Max level shown:
12025-04-30T06:43:35.897ZDEBGtest_utilization (init_with_steps): Running step start_crdb
22025-04-30T06:43:35.898ZDEBGtest_utilization: Starting CRDB
32025-04-30T06:43:35.947ZINFOtest_utilization: cockroach temporary directory: /var/tmp/omicron_tmp/.tmpWhmV7c
42025-04-30T06:43:35.947ZINFOtest_utilization: cockroach: copying from seed tarball (/var/tmp/omicron_tmp/crdb-base-build/f234518892194a05ad8767dc97f01a61a55baf1431942a314ce8b0f72b18ec2e.tar) to storage directory (/var/tmp/omicron_tmp/.tmpWhmV7c/data)
52025-04-30T06:43:35.992ZINFOtest_utilization: cockroach command line: cockroach start-single-node --insecure --http-addr=:0 --store=path=/var/tmp/omicron_tmp/.tmpWhmV7c/data,ballast-size=0 --listen-addr [::1]:0 --listening-url-file /var/tmp/omicron_tmp/.tmpWhmV7c/listen-url
62025-04-30T06:43:35.995ZINFOtest_utilization: cockroach environment: BUILDOMAT_JOB_ID=01JT2JWKGHF3JHFMW0JR13NSA6 BUILDOMAT_TASK_ID=4 CARGO=/home/build/.rustup/toolchains/1.86.0-x86_64-unknown-illumos/bin/cargo CARGO_HOME=/home/build/.cargo CARGO_INCREMENTAL=0 CARGO_MANIFEST_DIR=/work/oxidecomputer/omicron/nexus CARGO_PKG_AUTHORS= CARGO_PKG_DESCRIPTION= CARGO_PKG_HOMEPAGE= CARGO_PKG_LICENSE=MPL-2.0 CARGO_PKG_LICENSE_FILE= CARGO_PKG_NAME=omicron-nexus CARGO_PKG_REPOSITORY= CARGO_PKG_RUST_VERSION= CARGO_PKG_VERSION=0.1.0 CARGO_PKG_VERSION_MAJOR=0 CARGO_PKG_VERSION_MINOR=1 CARGO_PKG_VERSION_PATCH=0 CARGO_PKG_VERSION_PRE= CARGO_TERM_COLOR=always CI=true CRDB_SEED_TAR=/var/tmp/omicron_tmp/crdb-base-build/f234518892194a05ad8767dc97f01a61a55baf1431942a314ce8b0f72b18ec2e.tar GITHUB_REPOSITORY=oxidecomputer/omicron GITHUB_SHA=531912b3fc900900bc7bd18f5eac1ce1e22cb1cd GOTRACEBACK=crash HOME=/home/build LANG=en_US.UTF-8 LC_ALL=en_US.UTF-8 LD_LIBRARY_PATH=/work/oxidecomputer/omicron/target/debug/build/aws-lc-sys-6532f52db341a7cc/out/build/artifacts:/work/oxidecomputer/omicron/target/debug/build/blake3-403ee89fc24db8f8/out:/work/oxidecomputer/omicron/target/debug/build/bzip2-sys-aece66bfb5c8df92/out/lib:/work/oxidecomputer/omicron/target/debug/build/libgit2-sys-5fe90ecb72e1690f/out/build:/work/oxidecomputer/omicron/target/debug/build/ring-fb368fe91b2bd145/out:/work/oxidecomputer/omicron/target/debug/build/tofino-ffe28f7f2301d0c2/out:/work/oxidecomputer/omicron/target/debug/build/zstd-sys-a7fb19a1e17b70f6/out:/work/oxidecomputer/omicron/target/debug/deps:/work/oxidecomputer/omicron/target/debug:/home/build/.rustup/toolchains/1.86.0-x86_64-unknown-illumos/lib/rustlib/x86_64-unknown-illumos/lib:/home/build/.rustup/toolchains/1.86.0-x86_64-unknown-illumos/lib LOGNAME=build NEXTEST=1 NEXTEST_BIN_EXE_nexus=/work/oxidecomputer/omicron/target/debug/nexus NEXTEST_BIN_EXE_schema-updater=/work/oxidecomputer/omicron/target/debug/schema-updater NEXTEST_EXECUTION_MODE=process-per-test NEXTEST_LD_LIBRARY_PATH=/work/oxidecomputer/omicron/target/debug/build/aws-lc-sys-6532f52db341a7cc/out/build/artifacts:/work/oxidecomputer/omicron/target/debug/build/blake3-403ee89fc24db8f8/out:/work/oxidecomputer/omicron/target/debug/build/bzip2-sys-aece66bfb5c8df92/out/lib:/work/oxidecomputer/omicron/target/debug/build/libgit2-sys-5fe90ecb72e1690f/out/build:/work/oxidecomputer/omicron/target/debug/build/ring-fb368fe91b2bd145/out:/work/oxidecomputer/omicron/target/debug/build/tofino-ffe28f7f2301d0c2/out:/work/oxidecomputer/omicron/target/debug/build/zstd-sys-a7fb19a1e17b70f6/out:/work/oxidecomputer/omicron/target/debug/deps:/work/oxidecomputer/omicron/target/debug:/home/build/.rustup/toolchains/1.86.0-x86_64-unknown-illumos/lib/rustlib/x86_64-unknown-illumos/lib:/home/build/.rustup/toolchains/1.86.0-x86_64-unknown-illumos/lib NEXTEST_PROFILE=ci NEXTEST_RUN_ID=29735cc2-0883-4059-8ca2-1da53f01ede3 NEXTEST_TEST_GLOBAL_SLOT=4 NEXTEST_TEST_GROUP=@global NEXTEST_TEST_GROUP_SLOT=none OUT_DIR=/work/oxidecomputer/omicron/target/debug/build/omicron-nexus-553062f859dbbfa0/out PATH=/work/oxidecomputer/omicron/out/mgd/root/opt/oxide/mgd/bin:/work/oxidecomputer/omicron/out/dendrite-stub/bin:/work/oxidecomputer/omicron/out/clickhouse:/work/oxidecomputer/omicron/out/cockroachdb/bin:/home/build/.cargo/bin:/usr/bin:/bin:/usr/sbin:/sbin:/opt/ooce/bin:/opt/ooce/sbin PWD=/work/oxidecomputer/omicron RUSTC_BOOTSTRAP=1 RUSTDOCFLAGS=--document-private-items -D warnings RUSTFLAGS=-D warnings -C link-arg=-R/usr/platform/oxide/lib/amd64 RUSTUP_HOME=/home/build/.rustup RUSTUP_TOOLCHAIN=1.86.0-x86_64-unknown-illumos RUST_BACKTRACE=1 RUST_RECURSION_COUNT=1 SHLVL=1 SSL_CERT_DIR=/usr/ssl/certs SSL_CERT_FILE=/etc/ssl/cacert.pem TMPDIR=/var/tmp/omicron_tmp TZ=UTC USER=build _=/usr/bin/ptime __NEXTEST_ATTEMPT=1
72025-04-30T06:43:37.903ZINFOtest_utilization: cockroach pid: 27302
82025-04-30T06:43:37.903ZINFOtest_utilization: cockroach listen URL: postgresql://root@[::1]:65362/omicron?sslmode=disable
92025-04-30T06:43:37.904ZDEBGtest_utilization (init_with_steps): Running step start_clickhouse
102025-04-30T06:43:37.904ZDEBGtest_utilization: Starting Clickhouse
112025-04-30T06:43:38.500ZDEBGtest_utilization (init_with_steps): Running step start_gateway_switch0
122025-04-30T06:43:38.500ZDEBGtest_utilization: Starting Management Gateway
132025-04-30T06:43:38.625ZDEBGtest_utilization (init_with_steps): Running step start_dendrite_switch0
142025-04-30T06:43:38.625ZDEBGtest_utilization: Starting Dendrite for switch0
152025-04-30T06:43:38.713ZDEBGtest_utilization (init_with_steps): Running step start_mgd_switch0
162025-04-30T06:43:38.713ZDEBGtest_utilization: Starting mgd for switch0
172025-04-30T06:43:38.821ZDEBGtest_utilization: mgd port is 52102
182025-04-30T06:43:38.822ZDEBGtest_utilization (init_with_steps): Running step record_switch_dns
192025-04-30T06:43:38.827ZDEBGtest_utilization: Recording DNS for the switch zones sled_id = b6d65341-167c-41df-9b5c-41cded99c229 switch_location = switch0
202025-04-30T06:43:38.829ZDEBGtest_utilization (init_with_steps): Running step start_internal_dns
212025-04-30T06:43:38.831ZINFOtest_utilization (store): opening sled database path = /var/tmp/omicron_tmp/.tmpnpNXq6
222025-04-30T06:43:39.075ZINFOtest_utilization (store): pruning trees for generations newer than 0
232025-04-30T06:43:39.075ZINFOtest_utilization (store): pruning trees for generations older than 0 keep = 3
242025-04-30T06:43:39.075ZINFOtest_utilization (dns): DNS server bound to address kind = dns local_address = [::1]:61150
252025-04-30T06:43:39.077ZDEBGtest_utilization (http): registered endpoint kind = dns local_addr = [::1]:42391 method = GET path = /config versions = all
262025-04-30T06:43:39.077ZDEBGtest_utilization (http): registered endpoint kind = dns local_addr = [::1]:42391 method = PUT path = /config versions = all
272025-04-30T06:43:39.077ZINFOtest_utilization (http): listening kind = dns local_addr = [::1]:42391
282025-04-30T06:43:39.078ZDEBGtest_utilization (http): successfully registered DTrace USDT probes kind = dns local_addr = [::1]:42391
292025-04-30T06:43:39.078ZDEBGtest_utilization (init_with_steps): Running step start_external_dns
302025-04-30T06:43:39.078ZINFOtest_utilization (store): opening sled database path = /var/tmp/omicron_tmp/.tmpZwRf3H
312025-04-30T06:43:39.088ZINFOtest_utilization (store): pruning trees for generations newer than 0
322025-04-30T06:43:39.088ZINFOtest_utilization (store): pruning trees for generations older than 0 keep = 3
332025-04-30T06:43:39.088ZINFOtest_utilization (dns): DNS server bound to address kind = dns local_address = [::1]:59834
342025-04-30T06:43:39.088ZDEBGtest_utilization (http): registered endpoint kind = dns local_addr = [::1]:48132 method = GET path = /config versions = all
352025-04-30T06:43:39.088ZDEBGtest_utilization (http): registered endpoint kind = dns local_addr = [::1]:48132 method = PUT path = /config versions = all
362025-04-30T06:43:39.088ZINFOtest_utilization (http): listening kind = dns local_addr = [::1]:48132
372025-04-30T06:43:39.089ZDEBGtest_utilization (http): successfully registered DTrace USDT probes kind = dns local_addr = [::1]:48132
382025-04-30T06:43:39.090ZDEBGtest_utilization (init_with_steps): Running step start_nexus_internal
392025-04-30T06:43:39.090ZDEBGtest_utilization: Starting Nexus (internal API)
402025-04-30T06:43:39.090ZINFOtest_utilization: setting up nexus server
412025-04-30T06:43:39.093ZINFOtest_utilization (ServerContext): registering Oso class class = Action
422025-04-30T06:43:39.093ZINFOtest_utilization (ServerContext): registering Oso class class = AnyActor
432025-04-30T06:43:39.093ZINFOtest_utilization (ServerContext): registering Oso class class = AuthenticatedActor
442025-04-30T06:43:39.094ZINFOtest_utilization (ServerContext): registering Oso class class = BlueprintConfig
452025-04-30T06:43:39.094ZINFOtest_utilization (ServerContext): registering Oso class class = Database
462025-04-30T06:43:39.094ZINFOtest_utilization (ServerContext): registering Oso class class = DnsConfig
472025-04-30T06:43:39.094ZINFOtest_utilization (ServerContext): registering Oso class class = Fleet
482025-04-30T06:43:39.094ZINFOtest_utilization (ServerContext): registering Oso class class = Inventory
492025-04-30T06:43:39.094ZINFOtest_utilization (ServerContext): registering Oso class class = IpPoolList
502025-04-30T06:43:39.094ZINFOtest_utilization (ServerContext): registering Oso class class = ConsoleSessionList
512025-04-30T06:43:39.094ZINFOtest_utilization (ServerContext): registering Oso class class = DeviceAuthRequestList
522025-04-30T06:43:39.094ZINFOtest_utilization (ServerContext): registering Oso class class = SiloCertificateList
532025-04-30T06:43:39.094ZINFOtest_utilization (ServerContext): registering Oso class class = SiloIdentityProviderList
542025-04-30T06:43:39.095ZINFOtest_utilization (ServerContext): registering Oso class class = SiloUserList
552025-04-30T06:43:39.095ZINFOtest_utilization (ServerContext): registering Oso class class = TargetReleaseConfig
562025-04-30T06:43:39.095ZINFOtest_utilization (ServerContext): registering Oso class class = WebhookEventClassList
572025-04-30T06:43:39.095ZINFOtest_utilization (ServerContext): registering Oso class class = Project
582025-04-30T06:43:39.096ZINFOtest_utilization (ServerContext): registering Oso class class = Disk
592025-04-30T06:43:39.096ZINFOtest_utilization (ServerContext): registering Oso class class = Snapshot
602025-04-30T06:43:39.096ZINFOtest_utilization (ServerContext): registering Oso class class = ProjectImage
612025-04-30T06:43:39.096ZINFOtest_utilization (ServerContext): registering Oso class class = AffinityGroup
622025-04-30T06:43:39.097ZINFOtest_utilization (ServerContext): registering Oso class class = AntiAffinityGroup
632025-04-30T06:43:39.097ZINFOtest_utilization (ServerContext): registering Oso class class = Instance
642025-04-30T06:43:39.098ZINFOtest_utilization (ServerContext): registering Oso class class = IpPool
652025-04-30T06:43:39.098ZINFOtest_utilization (ServerContext): registering Oso class class = InstanceNetworkInterface
662025-04-30T06:43:39.098ZINFOtest_utilization (ServerContext): registering Oso class class = Vpc
672025-04-30T06:43:39.098ZINFOtest_utilization (ServerContext): registering Oso class class = VpcRouter
682025-04-30T06:43:39.098ZINFOtest_utilization (ServerContext): registering Oso class class = InternetGateway
692025-04-30T06:43:39.098ZINFOtest_utilization (ServerContext): registering Oso class class = InternetGatewayIpPool
702025-04-30T06:43:39.098ZINFOtest_utilization (ServerContext): registering Oso class class = InternetGatewayIpAddress
712025-04-30T06:43:39.098ZINFOtest_utilization (ServerContext): registering Oso class class = RouterRoute
722025-04-30T06:43:39.098ZINFOtest_utilization (ServerContext): registering Oso class class = VpcSubnet
732025-04-30T06:43:39.098ZINFOtest_utilization (ServerContext): registering Oso class class = FloatingIp
742025-04-30T06:43:39.099ZINFOtest_utilization (ServerContext): registering Oso class class = Image
752025-04-30T06:43:39.099ZINFOtest_utilization (ServerContext): registering Oso class class = SiloImage
762025-04-30T06:43:39.099ZINFOtest_utilization (ServerContext): registering Oso class class = AddressLot
772025-04-30T06:43:39.099ZINFOtest_utilization (ServerContext): registering Oso class class = Blueprint
782025-04-30T06:43:39.099ZINFOtest_utilization (ServerContext): registering Oso class class = LoopbackAddress
792025-04-30T06:43:39.099ZINFOtest_utilization (ServerContext): registering Oso class class = Certificate
802025-04-30T06:43:39.099ZINFOtest_utilization (ServerContext): registering Oso class class = ConsoleSession
812025-04-30T06:43:39.099ZINFOtest_utilization (ServerContext): registering Oso class class = DeviceAuthRequest
822025-04-30T06:43:39.099ZINFOtest_utilization (ServerContext): registering Oso class class = DeviceAccessToken
832025-04-30T06:43:39.099ZINFOtest_utilization (ServerContext): registering Oso class class = PhysicalDisk
842025-04-30T06:43:39.100ZINFOtest_utilization (ServerContext): registering Oso class class = Rack
852025-04-30T06:43:39.100ZINFOtest_utilization (ServerContext): registering Oso class class = RoleBuiltin
862025-04-30T06:43:39.100ZINFOtest_utilization (ServerContext): registering Oso class class = SshKey
872025-04-30T06:43:39.100ZINFOtest_utilization (ServerContext): registering Oso class class = Silo
882025-04-30T06:43:39.100ZINFOtest_utilization (ServerContext): registering Oso class class = SiloUser
892025-04-30T06:43:39.100ZINFOtest_utilization (ServerContext): registering Oso class class = SiloGroup
902025-04-30T06:43:39.100ZINFOtest_utilization (ServerContext): registering Oso class class = SupportBundle
912025-04-30T06:43:39.101ZINFOtest_utilization (ServerContext): registering Oso class class = IdentityProvider
922025-04-30T06:43:39.101ZINFOtest_utilization (ServerContext): registering Oso class class = SamlIdentityProvider
932025-04-30T06:43:39.101ZINFOtest_utilization (ServerContext): registering Oso class class = Sled
942025-04-30T06:43:39.101ZINFOtest_utilization (ServerContext): registering Oso class class = TufRepo
952025-04-30T06:43:39.101ZINFOtest_utilization (ServerContext): registering Oso class class = TufArtifact
962025-04-30T06:43:39.101ZINFOtest_utilization (ServerContext): registering Oso class class = WebhookEvent
972025-04-30T06:43:39.101ZINFOtest_utilization (ServerContext): registering Oso class class = WebhookReceiver
982025-04-30T06:43:39.101ZINFOtest_utilization (ServerContext): registering Oso class class = WebhookSecret
992025-04-30T06:43:39.101ZINFOtest_utilization (ServerContext): registering Oso class class = Zpool
1002025-04-30T06:43:39.101ZINFOtest_utilization (ServerContext): registering Oso class class = Service
1012025-04-30T06:43:39.102ZINFOtest_utilization (ServerContext): registering Oso class class = UserBuiltin
1022025-04-30T06:43:39.102ZINFOtest_utilization (ServerContext): full Oso configuration config = #\n# Oso configuration for Omicron\n# This file is augmented by generated snippets.\n#\n\n#\n# ACTOR TYPES AND BASIC RULES\n#\n\n# `AnyActor` includes both authenticated and unauthenticated users.\nactor AnyActor {}\n\n# An `AuthenticatedActor` has an identity in the system. All of our operations\n# today require that an actor be authenticated.\nactor AuthenticatedActor {}\n\n# For any resource, `actor` can perform action `action` on it if they're\n# authenticated and their role(s) give them the corresponding permission on that\n# resource.\nallow(actor: AnyActor, action: Action, resource) if\n actor.authenticated and\n has_permission(actor.authn_actor.unwrap(), action.to_perm(), resource);\n\n# Define role relationships\nhas_role(actor: AuthenticatedActor, role: String, resource: Resource)\n\tif resource.has_role(actor, role);\n\n#\n# ROLES AND PERMISSIONS IN THE FLEET/SILO/PROJECT HIERARCHY\n#\n# We define the following permissions for most resources in the system:\n#\n# - "create_child": required to create child resources (of any type)\n#\n# - "list_children": required to list child resources (of all types) of a\n# resource\n#\n# - "modify": required to modify or delete a resource\n#\n# - "read": required to read a resource\n#\n# We define the following predefined roles for only a few high-level resources:\n# the Fleet (see below), Silo, Organization, and Project. The specific roles\n# are oriented around intended use-cases:\n#\n# - "admin": has all permissions on the resource\n#\n# - "collaborator": has "read", "list_children", and "create_child", plus\n# the "admin" role for child resources. The idea is that if you're an\n# Organization Collaborator, you have full control over the Projects within\n# the Organization, but you cannot modify or delete the Organization itself.\n#\n# - "viewer": has "read" and "list_children" on a resource\n#\n# Below the Project level, permissions are granted via roles at the Project\n# level. For example, for someone to be able to create, modify, or delete any\n# Instances, they must be granted project.collaborator, which means they can\n# create, modify, or delete _all_ resources in the Project.\n#\n# The complete set of predefined roles:\n#\n# - fleet.admin (superuser for the whole system)\n# - fleet.collaborator (can manage Silos)\n# - fleet.viewer (can read most non-siloed resources in the system)\n# - silo.admin (superuser for the silo)\n# - silo.collaborator (can create and own Organizations)\n# - silo.viewer (can read most resources within the Silo)\n# - organization.admin (complete control over an organization)\n# - organization.collaborator (can manage Projects)\n# - organization.viewer (can read most resources within the Organization)\n# - project.admin (complete control over a Project)\n# - project.collaborator (can manage all resources within the Project)\n# - project.viewer (can read most resources within the Project)\n#\n# Outside the Silo/Organization/Project hierarchy, we (currently) treat most\n# resources as nested under Fleet or else a synthetic resource (see below). We\n# do not yet support role assignments on anything other than Fleet, Silo,\n# Organization, or Project.\n#\n\n# "Fleet" is a global singleton representing the whole system. The name comes\n# from the idea described in RFD 24, but it's not quite right. This probably\n# should be more like "Region" or "AvailabilityZone". The precise boundaries\n# have not yet been figured out.\nresource Fleet {\n\tpermissions = [\n\t "list_children",\n\t "modify",\n\t "read",\n\t "create_child",\n\t];\n\n\troles = [\n\t # Roles that can be attached by users\n\t "admin",\n\t "collaborator",\n\t "viewer",\n\n\t # Internal-only roles\n\t "external-authenticator"\n\t];\n\n\t# Roles implied by other roles on this resource\n\t"viewer" if "collaborator";\n\t"collaborator" if "admin";\n\n\t# Permissions granted directly by roles on this resource\n\t"list_children" if "viewer";\n\t"read" if "viewer";\n\t"create_child" if "collaborator";\n\t"modify" if "admin";\n}\n\n# For fleets specifically, roles can be conferred by roles on the user's Silo.\nhas_role(actor: AuthenticatedActor, role: String, _: Fleet) if\n\tsilo_role in actor.confers_fleet_role(role) and\n\thas_role(actor, silo_role, actor.silo.unwrap());\n\nresource Silo {\n\tpermissions = [\n\t "list_children",\n\t "modify",\n\t "read",\n\t "create_child",\n\t];\n\troles = [ "admin", "collaborator", "viewer" ];\n\n\t# Roles implied by other roles on this resource\n\t"viewer" if "collaborator";\n\t"collaborator" if "admin";\n\n\t# Permissions granted directly by roles on this resource\n\t"list_children" if "viewer";\n\t"read" if "viewer";\n\n\t"create_child" if "collaborator";\n\t"modify" if "admin";\n\n\t# Permissions implied by roles on this resource's parent (Fleet). Fleet\n\t# privileges allow a user to see and potentially administer the Silo,\n\t# but they do not give anyone permission to look at anything inside the\n\t# Silo. To achieve this, we use permission rules here. (If we granted\n\t# Fleet administrators _roles_ on the Silo, then those would cascade\n\t# into the Silo as well.)\n\trelations = { parent_fleet: Fleet };\n\t"read" if "viewer" on "parent_fleet";\n\t"modify" if "collaborator" on "parent_fleet";\n\n\t# external authenticator has to create silo users\n\t"list_children" if "external-authenticator" on "parent_fleet";\n\t"create_child" if "external-authenticator" on "parent_fleet";\n}\n\nhas_relation(fleet: Fleet, "parent_fleet", silo: Silo)\n\tif silo.fleet = fleet;\n\n# As a special case, all authenticated users can read their own Silo. That's\n# not quite the same as having the "viewer" role. For example, they cannot list\n# Organizations in the Silo.\n#\n# One reason this is necessary is because if an unprivileged user tries to\n# create an Organization using "POST /organizations", they should get back a 403\n# (which implies they're able to see /organizations, which is essentially seeing\n# the Silo itself) rather than a 404. This behavior isn't a hard constraint\n# (i.e., you could reasonably get a 404 for an API you're not allowed to call).\n# Nor is the implementation (i.e., we could special-case this endpoint somehow).\n# But granting this permission is the simplest way to keep this endpoint's\n# behavior consistent with the rest of the API.\n#\n# This rule is also used to determine if a user can list the identity providers\n# in the Silo (which they should be able to), since that's predicated on being\n# able to read the Silo.\n#\n# It's unclear what else would break if users couldn't see their own Silo.\nhas_permission(actor: AuthenticatedActor, "read", silo: Silo)\n\tif silo in actor.silo;\n\nresource Project {\n\tpermissions = [\n\t "list_children",\n\t "modify",\n\t "read",\n\t "create_child",\n\t];\n\troles = [ "admin", "collaborator", "viewer" ];\n\n\t# Roles implied by other roles on this resource\n\t"viewer" if "collaborator";\n\t"collaborator" if "admin";\n\n\t# Permissions granted directly by roles on this resource\n\t"list_children" if "viewer";\n\t"read" if "viewer";\n\t"create_child" if "collaborator";\n\t"modify" if "admin";\n\n\t# Roles implied by roles on this resource's parent (Silo)\n\trelations = { parent_silo: Silo };\n\t"admin" if "collaborator" on "parent_silo";\n\t"viewer" if "viewer" on "parent_silo";\n}\nhas_relation(silo: Silo, "parent_silo", project: Project)\n\tif project.silo = silo;\n\n#\n# GENERAL RESOURCES OUTSIDE THE SILO/PROJECT HIERARCHY\n#\n# Many resources use snippets of Polar generated by the `authz_resource!` Rust\n# macro. Some resources require custom Polar code. Those appear here.\n#\n\nresource Certificate {\n\tpermissions = [ "read", "modify" ];\n\trelations = { parent_silo: Silo, parent_fleet: Fleet };\n\n\t# Fleet-level and silo-level roles both grant privileges on certificates.\n\t"read" if "admin" on "parent_silo";\n\t"modify" if "admin" on "parent_silo";\n\t"read" if "admin" on "parent_fleet";\n\t"modify" if "admin" on "parent_fleet";\n}\nhas_relation(silo: Silo, "parent_silo", certificate: Certificate)\n\tif certificate.silo = silo;\nhas_relation(fleet: Fleet, "parent_fleet", certificate: Certificate)\n\tif certificate.silo.fleet = fleet;\n\nresource SiloUser {\n\tpermissions = [\n\t "list_children",\n\t "modify",\n\t "read",\n\t "create_child",\n\t];\n\n\t# Fleet and Silo administrators can manage a Silo's users. This is one\n\t# of the only areas of Silo configuration that Fleet Administrators have\n\t# permissions on.\n\trelations = { parent_silo: Silo, parent_fleet: Fleet };\n\t"list_children" if "read" on "parent_silo";\n\t"read" if "read" on "parent_silo";\n\t"modify" if "admin" on "parent_silo";\n\t"create_child" if "admin" on "parent_silo";\n\t"list_children" if "read" on "parent_fleet";\n\t"read" if "read" on "parent_fleet";\n\t"modify" if "admin" on "parent_fleet";\n\t"create_child" if "admin" on "parent_fleet";\n}\nhas_relation(silo: Silo, "parent_silo", user: SiloUser)\n\tif user.silo = silo;\nhas_relation(fleet: Fleet, "parent_fleet", user: SiloUser)\n\tif user.silo.fleet = fleet;\n\n# authenticated actors have all permissions on themselves\nhas_permission(actor: AuthenticatedActor, _perm: String, silo_user: SiloUser)\n if actor.equals_silo_user(silo_user);\n\nhas_permission(actor: AuthenticatedActor, "read", silo_user: SiloUser)\n if silo_user.silo in actor.silo;\n\nresource SiloGroup {\n\tpermissions = [\n\t "list_children",\n\t "modify",\n\t "read",\n\t "create_child",\n\t];\n\n\trelations = { parent_silo: Silo };\n\t"list_children" if "read" on "parent_silo";\n\t"read" if "read" on "parent_silo";\n\t"modify" if "admin" on "parent_silo";\n\t"create_child" if "admin" on "parent_silo";\n}\nhas_relation(silo: Silo, "parent_silo", group: SiloGroup)\n\tif group.silo = silo;\n\nresource SshKey {\n\tpermissions = [ "read", "modify" ];\n\trelations = { silo_user: SiloUser };\n\n\t"read" if "read" on "silo_user";\n\t"modify" if "modify" on "silo_user";\n}\nhas_relation(user: SiloUser, "silo_user", ssh_key: SshKey)\n\tif ssh_key.silo_user = user;\n\nresource IdentityProvider {\n\tpermissions = [\n\t "read",\n\t "modify",\n\t "create_child",\n\t "list_children",\n\t];\n\trelations = { parent_silo: Silo, parent_fleet: Fleet };\n\n\t# Silo-level roles grant privileges on identity providers.\n\t"read" if "viewer" on "parent_silo";\n\t"list_children" if "viewer" on "parent_silo";\n\t"modify" if "admin" on "parent_silo";\n\t"create_child" if "admin" on "parent_silo";\n\n\t# Fleet-level roles also grant privileges on identity providers.\n\t"read" if "viewer" on "parent_fleet";\n\t"list_children" if "viewer" on "parent_fleet";\n\t"modify" if "admin" on "parent_fleet";\n\t"create_child" if "admin" on "parent_fleet";\n}\nhas_relation(silo: Silo, "parent_silo", identity_provider: IdentityProvider)\n\tif identity_provider.silo = silo;\nhas_relation(fleet: Fleet, "parent_fleet", collection: IdentityProvider)\n\tif collection.silo.fleet = fleet;\n\nresource SamlIdentityProvider {\n\tpermissions = [\n\t "read",\n\t "modify",\n\t "create_child",\n\t "list_children",\n\t];\n\trelations = { parent_silo: Silo, parent_fleet: Fleet };\n\n\t# Silo-level roles grant privileges on identity providers.\n\t"read" if "viewer" on "parent_silo";\n\t"list_children" if "viewer" on "parent_silo";\n\t"modify" if "admin" on "parent_silo";\n\t"create_child" if "admin" on "parent_silo";\n\n\t# Fleet-level roles also grant privileges on identity providers.\n\t"read" if "viewer" on "parent_fleet";\n\t"list_children" if "viewer" on "parent_fleet";\n\t"modify" if "admin" on "parent_fleet";\n\t"create_child" if "admin" on "parent_fleet";\n}\nhas_relation(silo: Silo, "parent_silo", saml_identity_provider: SamlIdentityProvider)\n\tif saml_identity_provider.silo = silo;\nhas_relation(fleet: Fleet, "parent_fleet", collection: SamlIdentityProvider)\n\tif collection.silo.fleet = fleet;\n\n#\n# SYNTHETIC RESOURCES OUTSIDE THE SILO HIERARCHY\n#\n# The resources here do not correspond to anything that appears explicitly in\n# the API or is stored in the database. These are used either at the top level\n# of the API path (e.g., "/v1/system/ip-pools") or as an implementation detail of the system\n# (in the case of console sessions and "Database"). The policies are\n# either statically-defined in this file or driven by role assignments on the\n# Fleet. None of these resources defines their own roles.\n#\n\n# Describes the policy for reading and modifying DNS configuration\n# (both internal and external)\nresource DnsConfig {\n\tpermissions = [ "read", "modify" ];\n\trelations = { parent_fleet: Fleet };\n\t# "external-authenticator" requires these permissions because that's the\n\t# context that Nexus uses when creating and deleting Silos. These\n\t# operations necessarily need to read and modify DNS configuration.\n\t"read" if "external-authenticator" on "parent_fleet";\n\t"modify" if "external-authenticator" on "parent_fleet";\n\t# "admin" on the parent fleet also gets these permissions, primarily for\n\t# the test suite.\n\t"read" if "admin" on "parent_fleet";\n\t"modify" if "admin" on "parent_fleet";\n}\nhas_relation(fleet: Fleet, "parent_fleet", dns_config: DnsConfig)\n\tif dns_config.fleet = fleet;\n\n# Describes the policy for accessing blueprints\nresource BlueprintConfig {\n\tpermissions = [\n\t "list_children", # list blueprints\n\t "create_child", # create blueprint\n\t "read", # read the current target\n\t "modify", # change the current target\n\t];\n\n\trelations = { parent_fleet: Fleet };\n\t"create_child" if "admin" on "parent_fleet";\n\t"modify" if "admin" on "parent_fleet";\n\t"list_children" if "viewer" on "parent_fleet";\n\t"read" if "viewer" on "parent_fleet";\n}\nhas_relation(fleet: Fleet, "parent_fleet", list: BlueprintConfig)\n\tif list.fleet = fleet;\n\n# Describes the policy for accessing blueprints\nresource TargetReleaseConfig {\n\tpermissions = [\n\t "read", # read the current target release\n\t "modify", # change the current target release\n\t];\n\n\trelations = { parent_fleet: Fleet };\n\t"read" if "viewer" on "parent_fleet";\n\t"modify" if "admin" on "parent_fleet";\n}\nhas_relation(fleet: Fleet, "parent_fleet", resource: TargetReleaseConfig)\n\tif resource.fleet = fleet;\n\n# Describes the policy for reading and modifying low-level inventory\nresource Inventory {\n\tpermissions = [ "read", "modify" ];\n\trelations = { parent_fleet: Fleet };\n\t"read" if "viewer" on "parent_fleet";\n\t"modify" if "admin" on "parent_fleet";\n}\nhas_relation(fleet: Fleet, "parent_fleet", inventory: Inventory)\n\tif inventory.fleet = fleet;\n\n# Describes the policy for accessing "/v1/system/ip-pools" in the API\nresource IpPoolList {\n\tpermissions = [\n\t "list_children",\n\t "modify",\n\t "create_child",\n\t];\n\n\t# Fleet Administrators can create or modify the IP Pools list.\n\trelations = { parent_fleet: Fleet };\n\t"modify" if "admin" on "parent_fleet";\n\t"create_child" if "admin" on "parent_fleet";\n\n\t# Fleet Viewers can list IP Pools\n\t"list_children" if "viewer" on "parent_fleet";\n}\nhas_relation(fleet: Fleet, "parent_fleet", ip_pool_list: IpPoolList)\n\tif ip_pool_list.fleet = fleet;\n\n# Any authenticated user can create a child of a provided IP Pool.\n# This is necessary to use the pools when provisioning instances.\nhas_permission(actor: AuthenticatedActor, "create_child", ip_pool: IpPool)\n\tif silo in actor.silo and silo.fleet = ip_pool.fleet;\n\n# Describes the policy for creating and managing web console sessions.\nresource ConsoleSessionList {\n\tpermissions = [ "create_child" ];\n\trelations = { parent_fleet: Fleet };\n\t"create_child" if "external-authenticator" on "parent_fleet";\n}\nhas_relation(fleet: Fleet, "parent_fleet", collection: ConsoleSessionList)\n\tif collection.fleet = fleet;\n\n# Describes the policy for creating and managing device authorization requests.\nresource DeviceAuthRequestList {\n\tpermissions = [ "create_child" ];\n\trelations = { parent_fleet: Fleet };\n\t"create_child" if "external-authenticator" on "parent_fleet";\n}\nhas_relation(fleet: Fleet, "parent_fleet", collection: DeviceAuthRequestList)\n\tif collection.fleet = fleet;\n\n# Describes the policy for creating and managing Silo certificates\nresource SiloCertificateList {\n\tpermissions = [ "list_children", "create_child" ];\n\n\trelations = { parent_silo: Silo, parent_fleet: Fleet };\n\n\t# Both Fleet and Silo administrators can see and modify the Silo's\n\t# certificates.\n\t"list_children" if "admin" on "parent_silo";\n\t"list_children" if "admin" on "parent_fleet";\n\t"create_child" if "admin" on "parent_silo";\n\t"create_child" if "admin" on "parent_fleet";\n}\nhas_relation(silo: Silo, "parent_silo", collection: SiloCertificateList)\n\tif collection.silo = silo;\nhas_relation(fleet: Fleet, "parent_fleet", collection: SiloCertificateList)\n\tif collection.silo.fleet = fleet;\n\n# Describes the policy for creating and managing Silo identity providers\nresource SiloIdentityProviderList {\n\tpermissions = [ "list_children", "create_child" ];\n\n\trelations = { parent_silo: Silo, parent_fleet: Fleet };\n\n\t# Everyone who can read the Silo (which includes all the users in the\n\t# Silo) can see the identity providers in it.\n\t"list_children" if "read" on "parent_silo";\n\n\t# Fleet and Silo administrators can manage the Silo's identity provider\n\t# configuration. This is one of the only areas of Silo configuration\n\t# that Fleet Administrators have permissions on. This is also one of\n\t# the only cases where we need to look two levels up the hierarchy to\n\t# see if somebody has the right permission. For most other things,\n\t# permissions cascade down the hierarchy so we only need to look at the\n\t# parent.\n\t"create_child" if "admin" on "parent_silo";\n\t"create_child" if "admin" on "parent_fleet";\n}\nhas_relation(silo: Silo, "parent_silo", collection: SiloIdentityProviderList)\n\tif collection.silo = silo;\nhas_relation(fleet: Fleet, "parent_fleet", collection: SiloIdentityProviderList)\n\tif collection.silo.fleet = fleet;\n\n# Describes the policy for creating and managing Silo users (mostly intended for\n# API-managed users)\nresource SiloUserList {\n\tpermissions = [ "list_children", "create_child" ];\n\n\trelations = { parent_silo: Silo, parent_fleet: Fleet };\n\n\t# Everyone who can read the Silo (which includes all the users in the\n\t# Silo) can see the users in it.\n\t"list_children" if "read" on "parent_silo";\n\n\t# Fleet and Silo administrators can manage the Silo's users. This is\n\t# one of the only areas of Silo configuration that Fleet Administrators\n\t# have permissions on. This is also one of the few cases (so far) where\n\t# we need to look two levels up the hierarchy to see if somebody has the\n\t# right permission. For most other things, permissions cascade down the\n\t# hierarchy so we only need to look at the parent.\n\t"create_child" if "admin" on "parent_silo";\n\t"list_children" if "admin" on "parent_fleet";\n\t"create_child" if "admin" on "parent_fleet";\n}\nhas_relation(silo: Silo, "parent_silo", collection: SiloUserList)\n\tif collection.silo = silo;\nhas_relation(fleet: Fleet, "parent_fleet", collection: SiloUserList)\n\tif collection.silo.fleet = fleet;\n\n# These rules grants the external authenticator role the permissions it needs to\n# read silo users and modify their sessions. This is necessary for login to\n# work.\nhas_permission(actor: AuthenticatedActor, "read", silo: Silo)\n\tif has_role(actor, "external-authenticator", silo.fleet);\nhas_permission(actor: AuthenticatedActor, "read", user: SiloUser)\n\tif has_role(actor, "external-authenticator", user.silo.fleet);\nhas_permission(actor: AuthenticatedActor, "modify", user: SiloUser)\n\tif has_role(actor, "external-authenticator", user.silo.fleet);\nhas_permission(actor: AuthenticatedActor, "read", group: SiloGroup)\n\tif has_role(actor, "external-authenticator", group.silo.fleet);\nhas_permission(actor: AuthenticatedActor, "modify", group: SiloGroup)\n\tif has_role(actor, "external-authenticator", group.silo.fleet);\n\nhas_permission(actor: AuthenticatedActor, "read", session: ConsoleSession)\n\tif has_role(actor, "external-authenticator", session.fleet);\nhas_permission(actor: AuthenticatedActor, "modify", session: ConsoleSession)\n\tif has_role(actor, "external-authenticator", session.fleet);\n\n# All authenticated users can read and delete device authn requests because\n# by necessity these operations happen before we've figured out what user (or\n# even Silo) the device auth is associated with. Any user can claim a device\n# auth request with the right user code (that's how it works) -- it's the user\n# code and associated logic that prevents unauthorized access here.\nhas_permission(_actor: AuthenticatedActor, "read", _device_auth: DeviceAuthRequest);\nhas_permission(_actor: AuthenticatedActor, "modify", _device_auth: DeviceAuthRequest);\n\nhas_permission(actor: AuthenticatedActor, "read", device_token: DeviceAccessToken)\n\tif has_role(actor, "external-authenticator", device_token.fleet);\n\nhas_permission(actor: AuthenticatedActor, "read", identity_provider: IdentityProvider)\n\tif has_role(actor, "external-authenticator", identity_provider.silo.fleet);\n\nhas_permission(actor: AuthenticatedActor, "read", saml_identity_provider: SamlIdentityProvider)\n\tif has_role(actor, "external-authenticator", saml_identity_provider.silo.fleet);\n\n# Describes the policy for who can access the internal database.\nresource Database {\n\tpermissions = [\n\t # "query" is required to perform any query against the database,\n\t # whether a read or write query. This is checked when an operation\n\t # checks out a database connection from the connection pool.\n\t #\n\t # Any authenticated user gets this permission. There's generally\n\t # some other authz check involved in the database query. For\n\t # example, if you're querying the database to "read" a "Project", we\n\t # should also be checking that. So why do we do this at all? It's\n\t # a belt-and-suspenders measure so that if we somehow introduced an\n\t # unauthenticated code path that hits the database, it cannot be\n\t # used to DoS the database because we won't allow the operation to\n\t # make the query. (As long as the code path _is_ authenticated, we\n\t # can use throttling mechanisms to prevent DoS.)\n\t "query",\n\n\t # "modify" is required to populate database data that's delivered\n\t # with the system. It should also be required for schema changes,\n\t # when we support those. This is separate from "query" so that we\n\t # cannot accidentally invoke these code paths from API calls and\n\t # other general functions.\n\t "modify"\n\t];\n}\n\n# All authenticated users have the "query" permission on the database.\nhas_permission(_actor: AuthenticatedActor, "query", _resource: Database);\n\n# The "db-init" user is the only one with the "modify" permission.\nhas_permission(USER_DB_INIT: AuthenticatedActor, "modify", _resource: Database);\nhas_permission(USER_DB_INIT: AuthenticatedActor, "create_child", _resource: IpPoolList);\n# It also has "admin" on the internal silo to populate it with built-in resources.\n# TODO-completeness: actually limit to just internal silo and not all silos\nhas_role(USER_DB_INIT: AuthenticatedActor, "admin", _silo: Silo);\n\n# Allow the internal API admin permissions on all silos.\nhas_role(USER_INTERNAL_API: AuthenticatedActor, "admin", _silo: Silo);\n\nresource WebhookSecret {\n\tpermissions = [ "read", "modify" ];\n\trelations = { parent_webhook_receiver: WebhookReceiver };\n\n\t"read" if "read" on "parent_webhook_receiver";\n\t"modify" if "modify" on "parent_webhook_receiver";\n}\n\nhas_relation(rx: WebhookReceiver, "parent_webhook_receiver", secret: WebhookSecret)\n\tif secret.webhook_receiver = rx;\n\nresource WebhookEventClassList {\n\tpermissions = [ "list_children" ];\n\trelations = { parent_fleet: Fleet };\n\n\t"list_children" if "viewer" on "parent_fleet";\n}\n\nhas_relation(fleet: Fleet, "parent_fleet", collection: WebhookEventClassList)\n\tif collection.fleet = fleet;\n\n\n\n resource Disk {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n\n relations = { containing_project: Project };\n "list_children" if "viewer" on "containing_project";\n "read" if "viewer" on "containing_project";\n "modify" if "collaborator" on "containing_project";\n "create_child" if "collaborator" on "containing_project";\n }\n\n has_relation(parent: Project, "containing_project", child: Disk)\n if child.project = parent;\n \n\n resource Snapshot {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n\n relations = { containing_project: Project };\n "list_children" if "viewer" on "containing_project";\n "read" if "viewer" on "containing_project";\n "modify" if "collaborator" on "containing_project";\n "create_child" if "collaborator" on "containing_project";\n }\n\n has_relation(parent: Project, "containing_project", child: Snapshot)\n if child.project = parent;\n \n\n resource ProjectImage {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n\n relations = { containing_project: Project };\n "list_children" if "viewer" on "containing_project";\n "read" if "viewer" on "containing_project";\n "modify" if "collaborator" on "containing_project";\n "create_child" if "collaborator" on "containing_project";\n }\n\n has_relation(parent: Project, "containing_project", child: ProjectImage)\n if child.project = parent;\n \n\n resource AffinityGroup {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n\n relations = { containing_project: Project };\n "list_children" if "viewer" on "containing_project";\n "read" if "viewer" on "containing_project";\n "modify" if "collaborator" on "containing_project";\n "create_child" if "collaborator" on "containing_project";\n }\n\n has_relation(parent: Project, "containing_project", child: AffinityGroup)\n if child.project = parent;\n \n\n resource AntiAffinityGroup {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n\n relations = { containing_project: Project };\n "list_children" if "viewer" on "containing_project";\n "read" if "viewer" on "containing_project";\n "modify" if "collaborator" on "containing_project";\n "create_child" if "collaborator" on "containing_project";\n }\n\n has_relation(parent: Project, "containing_project", child: AntiAffinityGroup)\n if child.project = parent;\n \n\n resource Instance {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n\n relations = { containing_project: Project };\n "list_children" if "viewer" on "containing_project";\n "read" if "viewer" on "containing_project";\n "modify" if "collaborator" on "containing_project";\n "create_child" if "collaborator" on "containing_project";\n }\n\n has_relation(parent: Project, "containing_project", child: Instance)\n if child.project = parent;\n \n\n resource IpPool {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n \n relations = { parent_fleet: Fleet };\n "list_children" if "viewer" on "parent_fleet";\n "read" if "viewer" on "parent_fleet";\n "modify" if "admin" on "parent_fleet";\n "create_child" if "admin" on "parent_fleet";\n }\n has_relation(fleet: Fleet, "parent_fleet", child: IpPool)\n if child.fleet = fleet;\n \n\n resource InstanceNetworkInterface {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n\n relations = {\n containing_project: Project,\n parent: Instance\n };\n "list_children" if "viewer" on "containing_project";\n "read" if "viewer" on "containing_project";\n "modify" if "collaborator" on "containing_project";\n "create_child" if "collaborator" on "containing_project";\n }\n\n has_relation(project: Project, "containing_project", child: InstanceNetworkInterface)\n if has_relation(project, "containing_project", child.instance);\n\n has_relation(parent: Instance, "parent", child: InstanceNetworkInterface)\n if child.instance = parent;\n \n\n resource Vpc {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n\n relations = { containing_project: Project };\n "list_children" if "viewer" on "containing_project";\n "read" if "viewer" on "containing_project";\n "modify" if "collaborator" on "containing_project";\n "create_child" if "collaborator" on "containing_project";\n }\n\n has_relation(parent: Project, "containing_project", child: Vpc)\n if child.project = parent;\n \n\n resource VpcRouter {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n\n relations = {\n containing_project: Project,\n parent: Vpc\n };\n "list_children" if "viewer" on "containing_project";\n "read" if "viewer" on "containing_project";\n "modify" if "collaborator" on "containing_project";\n "create_child" if "collaborator" on "containing_project";\n }\n\n has_relation(project: Project, "containing_project", child: VpcRouter)\n if has_relation(project, "containing_project", child.vpc);\n\n has_relation(parent: Vpc, "parent", child: VpcRouter)\n if child.vpc = parent;\n \n\n resource InternetGateway {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n\n relations = {\n containing_project: Project,\n parent: Vpc\n };\n "list_children" if "viewer" on "containing_project";\n "read" if "viewer" on "containing_project";\n "modify" if "collaborator" on "containing_project";\n "create_child" if "collaborator" on "containing_project";\n }\n\n has_relation(project: Project, "containing_project", child: InternetGateway)\n if has_relation(project, "containing_project", child.vpc);\n\n has_relation(parent: Vpc, "parent", child: InternetGateway)\n if child.vpc = parent;\n \n\n resource InternetGatewayIpPool {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n\n relations = {\n containing_project: Project,\n parent: InternetGateway\n };\n "list_children" if "viewer" on "containing_project";\n "read" if "viewer" on "containing_project";\n "modify" if "collaborator" on "containing_project";\n "create_child" if "collaborator" on "containing_project";\n }\n\n has_relation(project: Project, "containing_project", child: InternetGatewayIpPool)\n if has_relation(project, "containing_project", child.internet_gateway);\n\n has_relation(parent: InternetGateway, "parent", child: InternetGatewayIpPool)\n if child.internet_gateway = parent;\n \n\n resource InternetGatewayIpAddress {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n\n relations = {\n containing_project: Project,\n parent: InternetGateway\n };\n "list_children" if "viewer" on "containing_project";\n "read" if "viewer" on "containing_project";\n "modify" if "collaborator" on "containing_project";\n "create_child" if "collaborator" on "containing_project";\n }\n\n has_relation(project: Project, "containing_project", child: InternetGatewayIpAddress)\n if has_relation(project, "containing_project", child.internet_gateway);\n\n has_relation(parent: InternetGateway, "parent", child: InternetGatewayIpAddress)\n if child.internet_gateway = parent;\n \n\n resource RouterRoute {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n\n relations = {\n containing_project: Project,\n parent: VpcRouter\n };\n "list_children" if "viewer" on "containing_project";\n "read" if "viewer" on "containing_project";\n "modify" if "collaborator" on "containing_project";\n "create_child" if "collaborator" on "containing_project";\n }\n\n has_relation(project: Project, "containing_project", child: RouterRoute)\n if has_relation(project, "containing_project", child.vpc_router);\n\n has_relation(parent: VpcRouter, "parent", child: RouterRoute)\n if child.vpc_router = parent;\n \n\n resource VpcSubnet {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n\n relations = {\n containing_project: Project,\n parent: Vpc\n };\n "list_children" if "viewer" on "containing_project";\n "read" if "viewer" on "containing_project";\n "modify" if "collaborator" on "containing_project";\n "create_child" if "collaborator" on "containing_project";\n }\n\n has_relation(project: Project, "containing_project", child: VpcSubnet)\n if has_relation(project, "containing_project", child.vpc);\n\n has_relation(parent: Vpc, "parent", child: VpcSubnet)\n if child.vpc = parent;\n \n\n resource FloatingIp {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n\n relations = { containing_project: Project };\n "list_children" if "viewer" on "containing_project";\n "read" if "viewer" on "containing_project";\n "modify" if "collaborator" on "containing_project";\n "create_child" if "collaborator" on "containing_project";\n }\n\n has_relation(parent: Project, "containing_project", child: FloatingIp)\n if child.project = parent;\n \n\n resource Image {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n\n relations = { containing_silo: Silo };\n "list_children" if "viewer" on "containing_silo";\n "read" if "viewer" on "containing_silo";\n "modify" if "collaborator" on "containing_silo";\n "create_child" if "collaborator" on "containing_silo";\n }\n\n has_relation(parent: Silo, "containing_silo", child: Image)\n if child.silo = parent;\n \n\n resource SiloImage {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n\n relations = { containing_silo: Silo };\n "list_children" if "viewer" on "containing_silo";\n "read" if "viewer" on "containing_silo";\n "modify" if "collaborator" on "containing_silo";\n "create_child" if "collaborator" on "containing_silo";\n }\n\n has_relation(parent: Silo, "containing_silo", child: SiloImage)\n if child.silo = parent;\n \n\n resource AddressLot {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n \n relations = { parent_fleet: Fleet };\n "list_children" if "viewer" on "parent_fleet";\n "read" if "viewer" on "parent_fleet";\n "modify" if "admin" on "parent_fleet";\n "create_child" if "admin" on "parent_fleet";\n }\n has_relation(fleet: Fleet, "parent_fleet", child: AddressLot)\n if child.fleet = fleet;\n \n\n resource Blueprint {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n \n relations = { parent_fleet: Fleet };\n "list_children" if "viewer" on "parent_fleet";\n "read" if "viewer" on "parent_fleet";\n "modify" if "admin" on "parent_fleet";\n "create_child" if "admin" on "parent_fleet";\n }\n has_relation(fleet: Fleet, "parent_fleet", child: Blueprint)\n if child.fleet = fleet;\n \n\n resource LoopbackAddress {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n \n relations = { parent_fleet: Fleet };\n "list_children" if "viewer" on "parent_fleet";\n "read" if "viewer" on "parent_fleet";\n "modify" if "admin" on "parent_fleet";\n "create_child" if "admin" on "parent_fleet";\n }\n has_relation(fleet: Fleet, "parent_fleet", child: LoopbackAddress)\n if child.fleet = fleet;\n \n\n\n resource ConsoleSession {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n \n relations = { parent_fleet: Fleet };\n "list_children" if "viewer" on "parent_fleet";\n "read" if "viewer" on "parent_fleet";\n "modify" if "admin" on "parent_fleet";\n "create_child" if "admin" on "parent_fleet";\n }\n has_relation(fleet: Fleet, "parent_fleet", child: ConsoleSession)\n if child.fleet = fleet;\n \n\n resource DeviceAuthRequest {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n \n relations = { parent_fleet: Fleet };\n "list_children" if "viewer" on "parent_fleet";\n "read" if "viewer" on "parent_fleet";\n "modify" if "admin" on "parent_fleet";\n "create_child" if "admin" on "parent_fleet";\n }\n has_relation(fleet: Fleet, "parent_fleet", child: DeviceAuthRequest)\n if child.fleet = fleet;\n \n\n resource DeviceAccessToken {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n \n relations = { parent_fleet: Fleet };\n "list_children" if "viewer" on "parent_fleet";\n "read" if "viewer" on "parent_fleet";\n "modify" if "admin" on "parent_fleet";\n "create_child" if "admin" on "parent_fleet";\n }\n has_relation(fleet: Fleet, "parent_fleet", child: DeviceAccessToken)\n if child.fleet = fleet;\n \n\n resource PhysicalDisk {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n \n relations = { parent_fleet: Fleet };\n "list_children" if "viewer" on "parent_fleet";\n "read" if "viewer" on "parent_fleet";\n "modify" if "admin" on "parent_fleet";\n "create_child" if "admin" on "parent_fleet";\n }\n has_relation(fleet: Fleet, "parent_fleet", child: PhysicalDisk)\n if child.fleet = fleet;\n \n\n resource Rack {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n \n relations = { parent_fleet: Fleet };\n "list_children" if "viewer" on "parent_fleet";\n "read" if "viewer" on "parent_fleet";\n "modify" if "admin" on "parent_fleet";\n "create_child" if "admin" on "parent_fleet";\n }\n has_relation(fleet: Fleet, "parent_fleet", child: Rack)\n if child.fleet = fleet;\n \n\n resource RoleBuiltin {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n \n relations = { parent_fleet: Fleet };\n "list_children" if "viewer" on "parent_fleet";\n "read" if "viewer" on "parent_fleet";\n "modify" if "admin" on "parent_fleet";\n "create_child" if "admin" on "parent_fleet";\n }\n has_relation(fleet: Fleet, "parent_fleet", child: RoleBuiltin)\n if child.fleet = fleet;\n \n\n\n\n\n\n resource SupportBundle {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n \n relations = { parent_fleet: Fleet };\n "list_children" if "viewer" on "parent_fleet";\n "read" if "viewer" on "parent_fleet";\n "modify" if "admin" on "parent_fleet";\n "create_child" if "admin" on "parent_fleet";\n }\n has_relation(fleet: Fleet, "parent_fleet", child: SupportBundle)\n if child.fleet = fleet;\n \n\n\n\n resource Sled {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n \n relations = { parent_fleet: Fleet };\n "list_children" if "viewer" on "parent_fleet";\n "read" if "viewer" on "parent_fleet";\n "modify" if "admin" on "parent_fleet";\n "create_child" if "admin" on "parent_fleet";\n }\n has_relation(fleet: Fleet, "parent_fleet", child: Sled)\n if child.fleet = fleet;\n \n\n resource TufRepo {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n \n relations = { parent_fleet: Fleet };\n "list_children" if "viewer" on "parent_fleet";\n "read" if "viewer" on "parent_fleet";\n "modify" if "admin" on "parent_fleet";\n "create_child" if "admin" on "parent_fleet";\n }\n has_relation(fleet: Fleet, "parent_fleet", child: TufRepo)\n if child.fleet = fleet;\n \n\n resource TufArtifact {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n \n relations = { parent_fleet: Fleet };\n "list_children" if "viewer" on "parent_fleet";\n "read" if "viewer" on "parent_fleet";\n "modify" if "admin" on "parent_fleet";\n "create_child" if "admin" on "parent_fleet";\n }\n has_relation(fleet: Fleet, "parent_fleet", child: TufArtifact)\n if child.fleet = fleet;\n \n\n resource WebhookEvent {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n \n relations = { parent_fleet: Fleet };\n "list_children" if "viewer" on "parent_fleet";\n "read" if "viewer" on "parent_fleet";\n "modify" if "admin" on "parent_fleet";\n "create_child" if "admin" on "parent_fleet";\n }\n has_relation(fleet: Fleet, "parent_fleet", child: WebhookEvent)\n if child.fleet = fleet;\n \n\n resource WebhookReceiver {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n \n relations = { parent_fleet: Fleet };\n "list_children" if "viewer" on "parent_fleet";\n "read" if "viewer" on "parent_fleet";\n "modify" if "admin" on "parent_fleet";\n "create_child" if "admin" on "parent_fleet";\n }\n has_relation(fleet: Fleet, "parent_fleet", child: WebhookReceiver)\n if child.fleet = fleet;\n \n\n\n resource Zpool {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n \n relations = { parent_fleet: Fleet };\n "list_children" if "viewer" on "parent_fleet";\n "read" if "viewer" on "parent_fleet";\n "modify" if "admin" on "parent_fleet";\n "create_child" if "admin" on "parent_fleet";\n }\n has_relation(fleet: Fleet, "parent_fleet", child: Zpool)\n if child.fleet = fleet;\n \n\n resource Service {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n \n relations = { parent_fleet: Fleet };\n "list_children" if "viewer" on "parent_fleet";\n "read" if "viewer" on "parent_fleet";\n "modify" if "admin" on "parent_fleet";\n "create_child" if "admin" on "parent_fleet";\n }\n has_relation(fleet: Fleet, "parent_fleet", child: Service)\n if child.fleet = fleet;\n \n\n resource UserBuiltin {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n \n relations = { parent_fleet: Fleet };\n "list_children" if "viewer" on "parent_fleet";\n "read" if "viewer" on "parent_fleet";\n "modify" if "admin" on "parent_fleet";\n "create_child" if "admin" on "parent_fleet";\n }\n has_relation(fleet: Fleet, "parent_fleet", child: UserBuiltin)\n if child.fleet = fleet;\n
1032025-04-30T06:43:39.156ZINFOtest_utilization (ServerContext): Setting up resolver using DNS address: [::1]:61150
1042025-04-30T06:43:39.156ZINFOtest_utilization (DnsResolver): new DNS resolver addresses = [[::1]:61150]
1052025-04-30T06:43:39.156ZINFOtest_utilization (ServerContext): Setting up qorb database pool from a single host url = PostgresConfigWithUrl {\n url_raw: "postgresql://root@[::1]:65362/omicron?sslmode=disable",\n config: Config {\n user: Some(\n "root",\n ),\n password: None,\n dbname: Some(\n "omicron",\n ),\n options: None,\n application_name: None,\n ssl_mode: Disable,\n host: [\n Tcp(\n "::1",\n ),\n ],\n hostaddr: [],\n port: [\n 65362,\n ],\n connect_timeout: None,\n tcp_user_timeout: None,\n keepalives: true,\n keepalives_idle: 7200s,\n keepalives_interval: None,\n keepalives_retries: None,\n target_session_attrs: Any,\n channel_binding: Prefer,\n load_balance_hosts: Disable,\n },\n}
1062025-04-30T06:43:39.158ZDEBGtest_utilization (ServerContext): registered USDT probes
1072025-04-30T06:43:39.274ZINFOtest_utilization (nexus): Database schema version is up to date desired_version = 140.0.0 found_version = 140.0.0
1082025-04-30T06:43:39.421ZDEBGtest_utilization (clickhouse-client): registered USDT probes id = 4ac57349-d8f3-4050-bc24-dc6bceac664d
1092025-04-30T06:43:39.757ZDEBGtest_utilization (dropshot_internal): registered endpoint local_addr = 127.0.0.1:32956 method = GET path = /bgtasks versions = all
1102025-04-30T06:43:39.758ZDEBGtest_utilization (dropshot_internal): registered endpoint local_addr = 127.0.0.1:32956 method = POST path = /bgtasks/activate versions = all
1112025-04-30T06:43:39.758ZDEBGtest_utilization (dropshot_internal): registered endpoint local_addr = 127.0.0.1:32956 method = GET path = /bgtasks/view/{bgtask_name} versions = all
1122025-04-30T06:43:39.758ZDEBGtest_utilization (dropshot_internal): registered endpoint local_addr = 127.0.0.1:32956 method = GET path = /clickhouse/policy versions = all
1132025-04-30T06:43:39.758ZDEBGtest_utilization (dropshot_internal): registered endpoint local_addr = 127.0.0.1:32956 method = POST path = /clickhouse/policy versions = all
1142025-04-30T06:43:39.758ZDEBGtest_utilization (dropshot_internal): registered endpoint local_addr = 127.0.0.1:32956 method = POST path = /crucible/0/upstairs/{upstairs_id}/downstairs/{downstairs_id}/stop-request versions = all
1152025-04-30T06:43:39.758ZDEBGtest_utilization (dropshot_internal): registered endpoint local_addr = 127.0.0.1:32956 method = POST path = /crucible/0/upstairs/{upstairs_id}/downstairs/{downstairs_id}/stopped versions = all
1162025-04-30T06:43:39.758ZDEBGtest_utilization (dropshot_internal): registered endpoint local_addr = 127.0.0.1:32956 method = POST path = /crucible/0/upstairs/{upstairs_id}/repair/{repair_id}/progress versions = all
1172025-04-30T06:43:39.758ZDEBGtest_utilization (dropshot_internal): registered endpoint local_addr = 127.0.0.1:32956 method = POST path = /crucible/0/upstairs/{upstairs_id}/repair-finish versions = all
1182025-04-30T06:43:39.758ZDEBGtest_utilization (dropshot_internal): registered endpoint local_addr = 127.0.0.1:32956 method = POST path = /crucible/0/upstairs/{upstairs_id}/repair-start versions = all
1192025-04-30T06:43:39.758ZDEBGtest_utilization (dropshot_internal): registered endpoint local_addr = 127.0.0.1:32956 method = POST path = /demo-saga versions = all
1202025-04-30T06:43:39.758ZDEBGtest_utilization (dropshot_internal): registered endpoint local_addr = 127.0.0.1:32956 method = POST path = /demo-saga/{demo_saga_id}/complete versions = all
1212025-04-30T06:43:39.758ZDEBGtest_utilization (dropshot_internal): registered endpoint local_addr = 127.0.0.1:32956 method = GET path = /deployment/blueprints/all versions = all
1222025-04-30T06:43:39.758ZDEBGtest_utilization (dropshot_internal): registered endpoint local_addr = 127.0.0.1:32956 method = DELETE path = /deployment/blueprints/all/{blueprint_id} versions = all
1232025-04-30T06:43:39.758ZDEBGtest_utilization (dropshot_internal): registered endpoint local_addr = 127.0.0.1:32956 method = GET path = /deployment/blueprints/all/{blueprint_id} versions = all
1242025-04-30T06:43:39.758ZDEBGtest_utilization (dropshot_internal): registered endpoint local_addr = 127.0.0.1:32956 method = POST path = /deployment/blueprints/import versions = all
1252025-04-30T06:43:39.758ZDEBGtest_utilization (dropshot_internal): registered endpoint local_addr = 127.0.0.1:32956 method = POST path = /deployment/blueprints/regenerate versions = all
1262025-04-30T06:43:39.758ZDEBGtest_utilization (dropshot_internal): registered endpoint local_addr = 127.0.0.1:32956 method = GET path = /deployment/blueprints/target versions = all
1272025-04-30T06:43:39.758ZDEBGtest_utilization (dropshot_internal): registered endpoint local_addr = 127.0.0.1:32956 method = POST path = /deployment/blueprints/target versions = all
1282025-04-30T06:43:39.758ZDEBGtest_utilization (dropshot_internal): registered endpoint local_addr = 127.0.0.1:32956 method = PUT path = /deployment/blueprints/target/enabled versions = all
1292025-04-30T06:43:39.758ZDEBGtest_utilization (dropshot_internal): registered endpoint local_addr = 127.0.0.1:32956 method = POST path = /disk/{disk_id}/remove-read-only-parent versions = all
1302025-04-30T06:43:39.758ZDEBGtest_utilization (dropshot_internal): registered endpoint local_addr = 127.0.0.1:32956 method = PUT path = /disks/{disk_id} versions = all
1312025-04-30T06:43:39.758ZDEBGtest_utilization (dropshot_internal): registered endpoint local_addr = 127.0.0.1:32956 method = GET path = /experimental/v1/system/support-bundles versions = all
1322025-04-30T06:43:39.758ZDEBGtest_utilization (dropshot_internal): registered endpoint local_addr = 127.0.0.1:32956 method = POST path = /experimental/v1/system/support-bundles versions = all
1332025-04-30T06:43:39.759ZDEBGtest_utilization (dropshot_internal): registered endpoint local_addr = 127.0.0.1:32956 method = DELETE path = /experimental/v1/system/support-bundles/{support_bundle} versions = all
1342025-04-30T06:43:39.759ZDEBGtest_utilization (dropshot_internal): registered endpoint local_addr = 127.0.0.1:32956 method = GET path = /experimental/v1/system/support-bundles/{support_bundle} versions = all
1352025-04-30T06:43:39.759ZDEBGtest_utilization (dropshot_internal): registered endpoint local_addr = 127.0.0.1:32956 method = GET path = /experimental/v1/system/support-bundles/{support_bundle}/download versions = all
1362025-04-30T06:43:39.759ZDEBGtest_utilization (dropshot_internal): registered endpoint local_addr = 127.0.0.1:32956 method = HEAD path = /experimental/v1/system/support-bundles/{support_bundle}/download versions = all
1372025-04-30T06:43:39.759ZDEBGtest_utilization (dropshot_internal): registered endpoint local_addr = 127.0.0.1:32956 method = GET path = /experimental/v1/system/support-bundles/{support_bundle}/download/{file} versions = all
1382025-04-30T06:43:39.759ZDEBGtest_utilization (dropshot_internal): registered endpoint local_addr = 127.0.0.1:32956 method = HEAD path = /experimental/v1/system/support-bundles/{support_bundle}/download/{file} versions = all
1392025-04-30T06:43:39.759ZDEBGtest_utilization (dropshot_internal): registered endpoint local_addr = 127.0.0.1:32956 method = GET path = /experimental/v1/system/support-bundles/{support_bundle}/index versions = all
1402025-04-30T06:43:39.759ZDEBGtest_utilization (dropshot_internal): registered endpoint local_addr = 127.0.0.1:32956 method = POST path = /instances/{instance_id}/migrate versions = all
1412025-04-30T06:43:39.759ZDEBGtest_utilization (dropshot_internal): registered endpoint local_addr = 127.0.0.1:32956 method = POST path = /metrics/collectors versions = all
1422025-04-30T06:43:39.759ZDEBGtest_utilization (dropshot_internal): registered endpoint local_addr = 127.0.0.1:32956 method = GET path = /metrics/collectors/{collector_id}/producers versions = all
1432025-04-30T06:43:39.759ZDEBGtest_utilization (dropshot_internal): registered endpoint local_addr = 127.0.0.1:32956 method = POST path = /metrics/producers versions = all
1442025-04-30T06:43:39.759ZDEBGtest_utilization (dropshot_internal): registered endpoint local_addr = 127.0.0.1:32956 method = GET path = /mgs-updates versions = all
1452025-04-30T06:43:39.759ZDEBGtest_utilization (dropshot_internal): registered endpoint local_addr = 127.0.0.1:32956 method = GET path = /nat/ipv4/changeset/{from_gen} versions = all
1462025-04-30T06:43:39.759ZDEBGtest_utilization (dropshot_internal): registered endpoint local_addr = 127.0.0.1:32956 method = GET path = /oximeter/read-policy versions = all
1472025-04-30T06:43:39.759ZDEBGtest_utilization (dropshot_internal): registered endpoint local_addr = 127.0.0.1:32956 method = POST path = /oximeter/read-policy versions = all
1482025-04-30T06:43:39.759ZDEBGtest_utilization (dropshot_internal): registered endpoint local_addr = 127.0.0.1:32956 method = POST path = /physical-disk/expunge versions = all
1492025-04-30T06:43:39.759ZDEBGtest_utilization (dropshot_internal): registered endpoint local_addr = 127.0.0.1:32956 method = GET path = /probes/{sled} versions = all
1502025-04-30T06:43:39.759ZDEBGtest_utilization (dropshot_internal): registered endpoint local_addr = 127.0.0.1:32956 method = PUT path = /racks/{rack_id}/initialization-complete versions = all
1512025-04-30T06:43:39.759ZDEBGtest_utilization (dropshot_internal): registered endpoint local_addr = 127.0.0.1:32956 method = GET path = /sagas versions = all
1522025-04-30T06:43:39.759ZDEBGtest_utilization (dropshot_internal): registered endpoint local_addr = 127.0.0.1:32956 method = GET path = /sagas/{saga_id} versions = all
1532025-04-30T06:43:39.759ZDEBGtest_utilization (dropshot_internal): registered endpoint local_addr = 127.0.0.1:32956 method = GET path = /sled-agents/{sled_id} versions = all
1542025-04-30T06:43:39.759ZDEBGtest_utilization (dropshot_internal): registered endpoint local_addr = 127.0.0.1:32956 method = POST path = /sled-agents/{sled_id} versions = all
1552025-04-30T06:43:39.759ZDEBGtest_utilization (dropshot_internal): registered endpoint local_addr = 127.0.0.1:32956 method = POST path = /sled-agents/{sled_id}/firewall-rules-update versions = all
1562025-04-30T06:43:39.759ZDEBGtest_utilization (dropshot_internal): registered endpoint local_addr = 127.0.0.1:32956 method = POST path = /sleds/add versions = all
1572025-04-30T06:43:39.759ZDEBGtest_utilization (dropshot_internal): registered endpoint local_addr = 127.0.0.1:32956 method = POST path = /sleds/expunge versions = all
1582025-04-30T06:43:39.759ZDEBGtest_utilization (dropshot_internal): registered endpoint local_addr = 127.0.0.1:32956 method = GET path = /sleds/uninitialized versions = all
1592025-04-30T06:43:39.759ZDEBGtest_utilization (dropshot_internal): registered endpoint local_addr = 127.0.0.1:32956 method = PUT path = /switch/{switch_id} versions = all
1602025-04-30T06:43:39.759ZDEBGtest_utilization (dropshot_internal): registered endpoint local_addr = 127.0.0.1:32956 method = GET path = /v1/ping versions = all
1612025-04-30T06:43:39.759ZDEBGtest_utilization (dropshot_internal): registered endpoint local_addr = 127.0.0.1:32956 method = PUT path = /vmms/{propolis_id} versions = all
1622025-04-30T06:43:39.759ZDEBGtest_utilization (dropshot_internal): registered endpoint local_addr = 127.0.0.1:32956 method = POST path = /volume/{volume_id}/remove-read-only-parent versions = all
1632025-04-30T06:43:39.760ZINFOtest_utilization (dropshot_internal): listening local_addr = 127.0.0.1:32956
1642025-04-30T06:43:39.760ZDEBGtest_utilization (dropshot_internal): successfully registered DTrace USDT probes local_addr = 127.0.0.1:32956
1652025-04-30T06:43:39.760ZINFOtest_utilization (SEC): SEC running sec_id = ac55d8d9-a298-4534-898b-beee1c9e4e52
1662025-04-30T06:43:39.793ZTRACtest_utilization (DataLoader): authorize begin action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
1672025-04-30T06:43:39.803ZDEBGtest_utilization (DataLoader): roles roles = RoleSet { roles: {} }
1682025-04-30T06:43:39.838ZDEBGtest_utilization (DataLoader): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
1692025-04-30T06:43:39.839ZDEBGtest_utilization (DataLoader): attempting to create built-in users
1702025-04-30T06:43:39.840ZTRACtest_utilization (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
1712025-04-30T06:43:39.840ZDEBGtest_utilization (DataLoader): roles roles = RoleSet { roles: {} }
1722025-04-30T06:43:39.844ZDEBGtest_utilization (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
1732025-04-30T06:43:39.845ZINFOtest_utilization (MgsUpdateDriver): starting MgsUpdateDriver
1742025-04-30T06:43:39.865ZTRACtest_utilization (dns): buffer buffer = Length: 79 (0x4f) bytes\n0000: d5 63 01 00 00 01 00 00 00 00 00 01 10 5f 63 72 .c..........._cr\n0010: 75 63 69 62 6c 65 2d 70 61 6e 74 72 79 04 5f 74 ucible-pantry._t\n0020: 63 70 0d 63 6f 6e 74 72 6f 6c 2d 70 6c 61 6e 65 cp.control-plane\n0030: 05 6f 78 69 64 65 08 69 6e 74 65 72 6e 61 6c 00 .oxide.internal.\n0040: 00 21 00 01 00 00 29 04 d0 00 00 00 00 00 00 .!....)........ kind = dns peer_addr = [::1]:57160 req_id = 6b35f761-62d0-45ea-b873-e424c4143fe4
1752025-04-30T06:43:39.865ZDEBGtest_utilization (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 54627,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_crucible-pantry._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_crucible-pantry._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 16,\n 95,\n 99,\n 114,\n 117,\n 99,\n 105,\n 98,\n 108,\n 101,\n 45,\n 112,\n 97,\n 110,\n 116,\n 114,\n 121,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:57160 req_id = 6b35f761-62d0-45ea-b873-e424c4143fe4
1762025-04-30T06:43:39.867ZERROtest_utilization (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 54627, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 1, }, query: WireQuery { query: LowerQuery { name: LowerName( Name("_crucible-pantry._tcp.control-plane.oxide.internal."), ), original: Query { name: Name("_crucible-pantry._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN, }, }, original: [ 16, 95, 99, 114, 117, 99, 105, 98, 108, 101, 45, 112, 97, 110, 116, 114, 121, 4, 95, 116, 99, 112, 13, 99, 111, 110, 116, 114, 111, 108, 45, 112, 108, 97, 110, 101, 5, 111, 120, 105, 100, 101, 8, 105, 110, 116, 101, 114, 110, 97, 108, 0, 0, 33, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: Some( Edns { rcode_high: 0, version: 0, dnssec_ok: false, max_payload: 1232, options: OPT { options: {}, }, }, ), } SERVFAIL: server is not authoritative for name: "_crucible-pantry._tcp.control-plane.oxide.internal." kind = dns peer_addr = [::1]:57160 req_id = 6b35f761-62d0-45ea-b873-e424c4143fe4
1772025-04-30T06:43:39.867ZINFOtest_utilization (DataLoader): created 5 built-in users
1782025-04-30T06:43:39.868ZTRACtest_utilization (DataLoader): authorize begin action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
1792025-04-30T06:43:39.868ZDEBGtest_utilization (DataLoader): roles roles = RoleSet { roles: {} }
1802025-04-30T06:43:39.871ZDEBGtest_utilization (DataLoader): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
1812025-04-30T06:43:39.871ZDEBGtest_utilization (DataLoader): attempting to create built-in roles
1822025-04-30T06:43:39.871ZTRACtest_utilization (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
1832025-04-30T06:43:39.871ZDEBGtest_utilization (DataLoader): roles roles = RoleSet { roles: {} }
1842025-04-30T06:43:39.872ZDEBGtest_utilization (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
1852025-04-30T06:43:39.893ZINFOtest_utilization (DataLoader): created 10 built-in roles
1862025-04-30T06:43:39.893ZTRACtest_utilization (DataLoader): authorize begin action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
1872025-04-30T06:43:39.893ZDEBGtest_utilization (DataLoader): roles roles = RoleSet { roles: {} }
1882025-04-30T06:43:39.899ZDEBGtest_utilization (DataLoader): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
1892025-04-30T06:43:39.899ZDEBGtest_utilization (DataLoader): attempting to create built-in role assignments
1902025-04-30T06:43:39.899ZTRACtest_utilization (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
1912025-04-30T06:43:39.900ZDEBGtest_utilization (DataLoader): roles roles = RoleSet { roles: {} }
1922025-04-30T06:43:39.900ZDEBGtest_utilization (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
1932025-04-30T06:43:40.314ZINFOtest_utilization (DataLoader): created 4 built-in role assignments
1942025-04-30T06:43:40.314ZTRACtest_utilization (DataLoader): authorize begin action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
1952025-04-30T06:43:40.314ZDEBGtest_utilization (DataLoader): roles roles = RoleSet { roles: {} }
1962025-04-30T06:43:40.316ZDEBGtest_utilization (DataLoader): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
1972025-04-30T06:43:40.316ZDEBGtest_utilization (DataLoader): attempting to create built-in silos
1982025-04-30T06:43:40.316ZTRACtest_utilization (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
1992025-04-30T06:43:40.316ZDEBGtest_utilization (DataLoader): roles roles = RoleSet { roles: {} }
2002025-04-30T06:43:40.316ZDEBGtest_utilization (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
2012025-04-30T06:43:40.534ZINFOtest_utilization (DataLoader): created 2 built-in silos
2022025-04-30T06:43:40.535ZTRACtest_utilization (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
2032025-04-30T06:43:40.535ZDEBGtest_utilization (DataLoader): roles roles = RoleSet { roles: {} }
2042025-04-30T06:43:40.543ZDEBGtest_utilization (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
2052025-04-30T06:43:40.556ZTRACtest_utilization (DataLoader): authorize begin action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
2062025-04-30T06:43:40.556ZDEBGtest_utilization (DataLoader): roles roles = RoleSet { roles: {} }
2072025-04-30T06:43:40.558ZDEBGtest_utilization (DataLoader): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
2082025-04-30T06:43:40.558ZDEBGtest_utilization (DataLoader): attempting to create built-in projects
2092025-04-30T06:43:40.558ZTRACtest_utilization (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
2102025-04-30T06:43:40.558ZDEBGtest_utilization (DataLoader): roles roles = RoleSet { roles: {} }
2112025-04-30T06:43:40.559ZDEBGtest_utilization (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
2122025-04-30T06:43:40.564ZTRACtest_utilization (DataLoader): authorize begin action = CreateChild actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }
2132025-04-30T06:43:40.565ZTRACtest_utilization (DataLoader): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. } resource_id = 001de000-5110-4000-8000-000000000001 resource_type = Silo
2142025-04-30T06:43:40.565ZTRACtest_utilization (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
2152025-04-30T06:43:40.565ZDEBGtest_utilization (DataLoader): roles roles = RoleSet { roles: {} }
2162025-04-30T06:43:40.565ZDEBGtest_utilization (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
2172025-04-30T06:43:40.618ZTRACtest_utilization (DataLoader): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
2182025-04-30T06:43:40.618ZTRACtest_utilization (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
2192025-04-30T06:43:40.618ZDEBGtest_utilization (DataLoader): roles roles = RoleSet { roles: {} }
2202025-04-30T06:43:40.619ZDEBGtest_utilization (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
2212025-04-30T06:43:40.621ZDEBGtest_utilization (DataLoader): roles roles = RoleSet { roles: {} }
2222025-04-30T06:43:40.626ZDEBGtest_utilization (DataLoader): authorize result action = CreateChild actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) } result = Ok(())
2232025-04-30T06:43:40.626ZTRACtest_utilization (DataLoader): authorize begin action = CreateChild actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }
2242025-04-30T06:43:40.626ZTRACtest_utilization (DataLoader): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. } resource_id = 001de000-5110-4000-8000-000000000001 resource_type = Silo
2252025-04-30T06:43:40.626ZTRACtest_utilization (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
2262025-04-30T06:43:40.626ZDEBGtest_utilization (DataLoader): roles roles = RoleSet { roles: {} }
2272025-04-30T06:43:40.626ZDEBGtest_utilization (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
2282025-04-30T06:43:40.636ZTRACtest_utilization (DataLoader): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
2292025-04-30T06:43:40.636ZTRACtest_utilization (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
2302025-04-30T06:43:40.636ZDEBGtest_utilization (DataLoader): roles roles = RoleSet { roles: {} }
2312025-04-30T06:43:40.636ZDEBGtest_utilization (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
2322025-04-30T06:43:40.639ZDEBGtest_utilization (DataLoader): roles roles = RoleSet { roles: {} }
2332025-04-30T06:43:40.642ZDEBGtest_utilization (DataLoader): authorize result action = CreateChild actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) } result = Ok(())
2342025-04-30T06:43:40.642ZTRACtest_utilization (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
2352025-04-30T06:43:40.642ZDEBGtest_utilization (DataLoader): roles roles = RoleSet { roles: {} }
2362025-04-30T06:43:40.642ZDEBGtest_utilization (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
2372025-04-30T06:43:40.668ZINFOtest_utilization (DataLoader): created built-in services project
2382025-04-30T06:43:40.668ZTRACtest_utilization (DataLoader): authorize begin action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
2392025-04-30T06:43:40.668ZDEBGtest_utilization (DataLoader): roles roles = RoleSet { roles: {} }
2402025-04-30T06:43:40.675ZDEBGtest_utilization (DataLoader): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
2412025-04-30T06:43:40.675ZDEBGtest_utilization (DataLoader): attempting to create built-in VPCs
2422025-04-30T06:43:40.675ZTRACtest_utilization (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
2432025-04-30T06:43:40.675ZDEBGtest_utilization (DataLoader): roles roles = RoleSet { roles: {} }
2442025-04-30T06:43:40.676ZDEBGtest_utilization (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
2452025-04-30T06:43:40.688ZTRACtest_utilization (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
2462025-04-30T06:43:40.688ZDEBGtest_utilization (DataLoader): roles roles = RoleSet { roles: {} }
2472025-04-30T06:43:40.689ZDEBGtest_utilization (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
2482025-04-30T06:43:40.695ZTRACtest_utilization (DataLoader): authorize begin action = CreateChild actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }
2492025-04-30T06:43:40.695ZTRACtest_utilization (DataLoader): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. } resource_id = 001de000-4401-4000-8000-000000000000 resource_type = Project
2502025-04-30T06:43:40.705ZTRACtest_utilization (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
2512025-04-30T06:43:40.705ZDEBGtest_utilization (DataLoader): roles roles = RoleSet { roles: {} }
2522025-04-30T06:43:40.707ZDEBGtest_utilization (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
2532025-04-30T06:43:40.775ZTRACtest_utilization (DataLoader): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. } resource_id = 001de000-5110-4000-8000-000000000001 resource_type = Silo
2542025-04-30T06:43:40.775ZTRACtest_utilization (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
2552025-04-30T06:43:40.775ZDEBGtest_utilization (DataLoader): roles roles = RoleSet { roles: {} }
2562025-04-30T06:43:40.776ZDEBGtest_utilization (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
2572025-04-30T06:43:40.779ZTRACtest_utilization (DataLoader): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
2582025-04-30T06:43:40.780ZTRACtest_utilization (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
2592025-04-30T06:43:40.780ZDEBGtest_utilization (DataLoader): roles roles = RoleSet { roles: {} }
2602025-04-30T06:43:40.780ZDEBGtest_utilization (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
2612025-04-30T06:43:40.784ZDEBGtest_utilization (DataLoader): roles roles = RoleSet { roles: {} }
2622025-04-30T06:43:40.789ZDEBGtest_utilization (DataLoader): authorize result action = CreateChild actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) } result = Ok(())
2632025-04-30T06:43:40.789ZTRACtest_utilization (DataLoader): successful lookup of siloed resource "Project" using built-in user
2642025-04-30T06:43:40.790ZTRACtest_utilization (DataLoader): authorize begin action = CreateChild actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }
2652025-04-30T06:43:40.790ZTRACtest_utilization (DataLoader): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. } resource_id = 001de000-4401-4000-8000-000000000000 resource_type = Project
2662025-04-30T06:43:40.790ZTRACtest_utilization (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
2672025-04-30T06:43:40.790ZDEBGtest_utilization (DataLoader): roles roles = RoleSet { roles: {} }
2682025-04-30T06:43:40.790ZDEBGtest_utilization (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
2692025-04-30T06:43:40.792ZTRACtest_utilization (DataLoader): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. } resource_id = 001de000-5110-4000-8000-000000000001 resource_type = Silo
2702025-04-30T06:43:40.792ZTRACtest_utilization (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
2712025-04-30T06:43:40.792ZDEBGtest_utilization (DataLoader): roles roles = RoleSet { roles: {} }
2722025-04-30T06:43:40.793ZDEBGtest_utilization (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
2732025-04-30T06:43:40.795ZTRACtest_utilization (DataLoader): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
2742025-04-30T06:43:40.795ZTRACtest_utilization (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
2752025-04-30T06:43:40.795ZDEBGtest_utilization (DataLoader): roles roles = RoleSet { roles: {} }
2762025-04-30T06:43:40.796ZDEBGtest_utilization (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
2772025-04-30T06:43:40.798ZDEBGtest_utilization (DataLoader): roles roles = RoleSet { roles: {} }
2782025-04-30T06:43:40.832ZDEBGtest_utilization (DataLoader): authorize result action = CreateChild actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) } result = Ok(())
2792025-04-30T06:43:40.832ZTRACtest_utilization (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
2802025-04-30T06:43:40.832ZDEBGtest_utilization (DataLoader): roles roles = RoleSet { roles: {} }
2812025-04-30T06:43:40.833ZDEBGtest_utilization (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
2822025-04-30T06:43:40.854ZTRACtest_utilization (DataLoader): authorize begin action = CreateChild actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }
2832025-04-30T06:43:40.858ZTRACtest_utilization (DataLoader): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. } resource_id = 001de000-4401-4000-8000-000000000000 resource_type = Project
2842025-04-30T06:43:40.872ZTRACtest_utilization (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
2852025-04-30T06:43:40.885ZDEBGtest_utilization (DataLoader): roles roles = RoleSet { roles: {} }
2862025-04-30T06:43:40.890ZDEBGtest_utilization (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
2872025-04-30T06:43:40.899ZTRACtest_utilization (DataLoader): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. } resource_id = 001de000-5110-4000-8000-000000000001 resource_type = Silo
2882025-04-30T06:43:40.902ZTRACtest_utilization (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
2892025-04-30T06:43:40.902ZDEBGtest_utilization (DataLoader): roles roles = RoleSet { roles: {} }
2902025-04-30T06:43:40.902ZDEBGtest_utilization (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
2912025-04-30T06:43:40.908ZTRACtest_utilization (DataLoader): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
2922025-04-30T06:43:40.909ZTRACtest_utilization (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
2932025-04-30T06:43:40.909ZDEBGtest_utilization (DataLoader): roles roles = RoleSet { roles: {} }
2942025-04-30T06:43:40.910ZDEBGtest_utilization (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
2952025-04-30T06:43:40.921ZDEBGtest_utilization (DataLoader): roles roles = RoleSet { roles: {} }
2962025-04-30T06:43:40.932ZDEBGtest_utilization (DataLoader): authorize result action = CreateChild actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") } result = Ok(())
2972025-04-30T06:43:40.933ZTRACtest_utilization (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
2982025-04-30T06:43:40.933ZDEBGtest_utilization (DataLoader): roles roles = RoleSet { roles: {} }
2992025-04-30T06:43:40.934ZDEBGtest_utilization (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
3002025-04-30T06:43:40.948ZTRACtest_utilization (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
3012025-04-30T06:43:40.948ZDEBGtest_utilization (DataLoader): roles roles = RoleSet { roles: {} }
3022025-04-30T06:43:40.949ZDEBGtest_utilization (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
3032025-04-30T06:43:40.974ZTRACtest_utilization (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
3042025-04-30T06:43:40.975ZDEBGtest_utilization (DataLoader): roles roles = RoleSet { roles: {} }
3052025-04-30T06:43:40.976ZDEBGtest_utilization (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
3062025-04-30T06:43:40.991ZTRACtest_utilization (DataLoader): authorize begin action = CreateChild actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }
3072025-04-30T06:43:40.997ZTRACtest_utilization (DataLoader): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. } resource_id = 001de000-4401-4000-8000-000000000000 resource_type = Project
3082025-04-30T06:43:40.997ZTRACtest_utilization (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
3092025-04-30T06:43:40.997ZDEBGtest_utilization (DataLoader): roles roles = RoleSet { roles: {} }
3102025-04-30T06:43:40.998ZDEBGtest_utilization (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
3112025-04-30T06:43:41.009ZTRACtest_utilization (DataLoader): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. } resource_id = 001de000-5110-4000-8000-000000000001 resource_type = Silo
3122025-04-30T06:43:41.009ZTRACtest_utilization (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
3132025-04-30T06:43:41.009ZDEBGtest_utilization (DataLoader): roles roles = RoleSet { roles: {} }
3142025-04-30T06:43:41.010ZDEBGtest_utilization (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
3152025-04-30T06:43:41.027ZTRACtest_utilization (DataLoader): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
3162025-04-30T06:43:41.027ZTRACtest_utilization (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
3172025-04-30T06:43:41.027ZDEBGtest_utilization (DataLoader): roles roles = RoleSet { roles: {} }
3182025-04-30T06:43:41.027ZDEBGtest_utilization (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
3192025-04-30T06:43:41.031ZDEBGtest_utilization (DataLoader): roles roles = RoleSet { roles: {} }
3202025-04-30T06:43:41.038ZDEBGtest_utilization (DataLoader): authorize result action = CreateChild actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") } result = Ok(())
3212025-04-30T06:43:41.038ZTRACtest_utilization (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
3222025-04-30T06:43:41.038ZDEBGtest_utilization (DataLoader): roles roles = RoleSet { roles: {} }
3232025-04-30T06:43:41.038ZDEBGtest_utilization (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
3242025-04-30T06:43:41.094ZTRACtest_utilization (DataLoader): authorize begin action = CreateChild actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = VpcRouter { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-074c-4000-8000-000000000001, lookup_type: ById(001de000-074c-4000-8000-000000000001) }
3252025-04-30T06:43:41.094ZTRACtest_utilization (DataLoader): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. } resource_id = 001de000-4401-4000-8000-000000000000 resource_type = Project
3262025-04-30T06:43:41.094ZTRACtest_utilization (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
3272025-04-30T06:43:41.094ZDEBGtest_utilization (DataLoader): roles roles = RoleSet { roles: {} }
3282025-04-30T06:43:41.095ZDEBGtest_utilization (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
3292025-04-30T06:43:41.107ZTRACtest_utilization (DataLoader): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. } resource_id = 001de000-5110-4000-8000-000000000001 resource_type = Silo
3302025-04-30T06:43:41.108ZTRACtest_utilization (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
3312025-04-30T06:43:41.108ZDEBGtest_utilization (DataLoader): roles roles = RoleSet { roles: {} }
3322025-04-30T06:43:41.108ZDEBGtest_utilization (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
3332025-04-30T06:43:41.125ZTRACtest_utilization (DataLoader): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
3342025-04-30T06:43:41.125ZTRACtest_utilization (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
3352025-04-30T06:43:41.125ZDEBGtest_utilization (DataLoader): roles roles = RoleSet { roles: {} }
3362025-04-30T06:43:41.125ZDEBGtest_utilization (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
3372025-04-30T06:43:41.128ZDEBGtest_utilization (DataLoader): roles roles = RoleSet { roles: {} }
3382025-04-30T06:43:41.135ZDEBGtest_utilization (DataLoader): authorize result action = CreateChild actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = VpcRouter { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-074c-4000-8000-000000000001, lookup_type: ById(001de000-074c-4000-8000-000000000001) } result = Ok(())
3392025-04-30T06:43:41.135ZTRACtest_utilization (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
3402025-04-30T06:43:41.135ZDEBGtest_utilization (DataLoader): roles roles = RoleSet { roles: {} }
3412025-04-30T06:43:41.135ZDEBGtest_utilization (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
3422025-04-30T06:43:41.237ZTRACtest_utilization (DataLoader): authorize begin action = CreateChild actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = VpcRouter { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-074c-4000-8000-000000000001, lookup_type: ById(001de000-074c-4000-8000-000000000001) }
3432025-04-30T06:43:41.237ZTRACtest_utilization (DataLoader): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. } resource_id = 001de000-4401-4000-8000-000000000000 resource_type = Project
3442025-04-30T06:43:41.237ZTRACtest_utilization (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
3452025-04-30T06:43:41.237ZDEBGtest_utilization (DataLoader): roles roles = RoleSet { roles: {} }
3462025-04-30T06:43:41.238ZDEBGtest_utilization (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
3472025-04-30T06:43:41.241ZTRACtest_utilization (DataLoader): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. } resource_id = 001de000-5110-4000-8000-000000000001 resource_type = Silo
3482025-04-30T06:43:41.241ZTRACtest_utilization (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
3492025-04-30T06:43:41.241ZDEBGtest_utilization (DataLoader): roles roles = RoleSet { roles: {} }
3502025-04-30T06:43:41.242ZDEBGtest_utilization (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
3512025-04-30T06:43:41.275ZTRACtest_utilization (DataLoader): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
3522025-04-30T06:43:41.275ZTRACtest_utilization (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
3532025-04-30T06:43:41.275ZDEBGtest_utilization (DataLoader): roles roles = RoleSet { roles: {} }
3542025-04-30T06:43:41.280ZDEBGtest_utilization (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
3552025-04-30T06:43:41.284ZDEBGtest_utilization (DataLoader): roles roles = RoleSet { roles: {} }
3562025-04-30T06:43:41.291ZDEBGtest_utilization (DataLoader): authorize result action = CreateChild actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = VpcRouter { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-074c-4000-8000-000000000001, lookup_type: ById(001de000-074c-4000-8000-000000000001) } result = Ok(())
3572025-04-30T06:43:41.291ZTRACtest_utilization (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
3582025-04-30T06:43:41.291ZDEBGtest_utilization (DataLoader): roles roles = RoleSet { roles: {} }
3592025-04-30T06:43:41.291ZDEBGtest_utilization (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
3602025-04-30T06:43:41.308ZDEBGtest_utilization (DataLoader): attempting to create built-in VPC firewall rules
3612025-04-30T06:43:41.308ZTRACtest_utilization (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
3622025-04-30T06:43:41.308ZDEBGtest_utilization (DataLoader): roles roles = RoleSet { roles: {} }
3632025-04-30T06:43:41.309ZDEBGtest_utilization (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
3642025-04-30T06:43:41.336ZTRACtest_utilization (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
3652025-04-30T06:43:41.336ZDEBGtest_utilization (DataLoader): roles roles = RoleSet { roles: {} }
3662025-04-30T06:43:41.337ZDEBGtest_utilization (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
3672025-04-30T06:43:41.347ZTRACtest_utilization (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
3682025-04-30T06:43:41.347ZDEBGtest_utilization (DataLoader): roles roles = RoleSet { roles: {} }
3692025-04-30T06:43:41.349ZDEBGtest_utilization (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
3702025-04-30T06:43:41.358ZTRACtest_utilization (DataLoader): authorize begin action = CreateChild actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) }
3712025-04-30T06:43:41.358ZTRACtest_utilization (DataLoader): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. } resource_id = 001de000-4401-4000-8000-000000000000 resource_type = Project
3722025-04-30T06:43:41.358ZTRACtest_utilization (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
3732025-04-30T06:43:41.358ZDEBGtest_utilization (DataLoader): roles roles = RoleSet { roles: {} }
3742025-04-30T06:43:41.360ZDEBGtest_utilization (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
3752025-04-30T06:43:41.372ZTRACtest_utilization (DataLoader): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. } resource_id = 001de000-5110-4000-8000-000000000001 resource_type = Silo
3762025-04-30T06:43:41.372ZTRACtest_utilization (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
3772025-04-30T06:43:41.372ZDEBGtest_utilization (DataLoader): roles roles = RoleSet { roles: {} }
3782025-04-30T06:43:41.373ZDEBGtest_utilization (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
3792025-04-30T06:43:41.385ZTRACtest_utilization (DataLoader): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
3802025-04-30T06:43:41.386ZTRACtest_utilization (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
3812025-04-30T06:43:41.386ZDEBGtest_utilization (DataLoader): roles roles = RoleSet { roles: {} }
3822025-04-30T06:43:41.386ZDEBGtest_utilization (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
3832025-04-30T06:43:41.389ZDEBGtest_utilization (DataLoader): roles roles = RoleSet { roles: {} }
3842025-04-30T06:43:41.397ZDEBGtest_utilization (DataLoader): authorize result action = CreateChild actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
3852025-04-30T06:43:41.397ZTRACtest_utilization (DataLoader): successful lookup of siloed resource "Vpc" using built-in user
3862025-04-30T06:43:41.397ZTRACtest_utilization (DataLoader): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) }
3872025-04-30T06:43:41.397ZTRACtest_utilization (DataLoader): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. } resource_id = 001de000-4401-4000-8000-000000000000 resource_type = Project
3882025-04-30T06:43:41.397ZTRACtest_utilization (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
3892025-04-30T06:43:41.398ZDEBGtest_utilization (DataLoader): roles roles = RoleSet { roles: {} }
3902025-04-30T06:43:41.398ZDEBGtest_utilization (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
3912025-04-30T06:43:41.401ZTRACtest_utilization (DataLoader): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. } resource_id = 001de000-5110-4000-8000-000000000001 resource_type = Silo
3922025-04-30T06:43:41.401ZTRACtest_utilization (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
3932025-04-30T06:43:41.401ZDEBGtest_utilization (DataLoader): roles roles = RoleSet { roles: {} }
3942025-04-30T06:43:41.402ZDEBGtest_utilization (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
3952025-04-30T06:43:41.404ZTRACtest_utilization (DataLoader): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
3962025-04-30T06:43:41.404ZTRACtest_utilization (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
3972025-04-30T06:43:41.404ZDEBGtest_utilization (DataLoader): roles roles = RoleSet { roles: {} }
3982025-04-30T06:43:41.405ZDEBGtest_utilization (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
3992025-04-30T06:43:41.411ZDEBGtest_utilization (DataLoader): roles roles = RoleSet { roles: {} }
4002025-04-30T06:43:41.419ZDEBGtest_utilization (DataLoader): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
4012025-04-30T06:43:41.419ZTRACtest_utilization (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
4022025-04-30T06:43:41.419ZDEBGtest_utilization (DataLoader): roles roles = RoleSet { roles: {} }
4032025-04-30T06:43:41.420ZDEBGtest_utilization (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
4042025-04-30T06:43:41.441ZTRACtest_utilization (DataLoader): authorize begin action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) }
4052025-04-30T06:43:41.441ZTRACtest_utilization (DataLoader): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. } resource_id = 001de000-4401-4000-8000-000000000000 resource_type = Project
4062025-04-30T06:43:41.441ZTRACtest_utilization (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
4072025-04-30T06:43:41.441ZDEBGtest_utilization (DataLoader): roles roles = RoleSet { roles: {} }
4082025-04-30T06:43:41.441ZDEBGtest_utilization (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
4092025-04-30T06:43:41.450ZTRACtest_utilization (DataLoader): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. } resource_id = 001de000-5110-4000-8000-000000000001 resource_type = Silo
4102025-04-30T06:43:41.450ZTRACtest_utilization (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
4112025-04-30T06:43:41.450ZDEBGtest_utilization (DataLoader): roles roles = RoleSet { roles: {} }
4122025-04-30T06:43:41.451ZDEBGtest_utilization (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
4132025-04-30T06:43:41.453ZTRACtest_utilization (DataLoader): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
4142025-04-30T06:43:41.453ZTRACtest_utilization (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
4152025-04-30T06:43:41.453ZDEBGtest_utilization (DataLoader): roles roles = RoleSet { roles: {} }
4162025-04-30T06:43:41.454ZDEBGtest_utilization (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
4172025-04-30T06:43:41.456ZDEBGtest_utilization (DataLoader): roles roles = RoleSet { roles: {} }
4182025-04-30T06:43:41.474ZDEBGtest_utilization (DataLoader): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
4192025-04-30T06:43:41.474ZTRACtest_utilization (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
4202025-04-30T06:43:41.474ZDEBGtest_utilization (DataLoader): roles roles = RoleSet { roles: {} }
4212025-04-30T06:43:41.475ZDEBGtest_utilization (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
4222025-04-30T06:43:41.563ZINFOtest_utilization (DataLoader): created built-in VPC firewall rules
4232025-04-30T06:43:41.563ZDEBGtest_utilization (DataLoader): attempting to create built-in VPC Subnets
4242025-04-30T06:43:41.563ZTRACtest_utilization (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
4252025-04-30T06:43:41.563ZDEBGtest_utilization (DataLoader): roles roles = RoleSet { roles: {} }
4262025-04-30T06:43:41.563ZDEBGtest_utilization (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
4272025-04-30T06:43:41.585ZTRACtest_utilization (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
4282025-04-30T06:43:41.585ZDEBGtest_utilization (DataLoader): roles roles = RoleSet { roles: {} }
4292025-04-30T06:43:41.586ZDEBGtest_utilization (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
4302025-04-30T06:43:41.601ZTRACtest_utilization (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
4312025-04-30T06:43:41.601ZDEBGtest_utilization (DataLoader): roles roles = RoleSet { roles: {} }
4322025-04-30T06:43:41.602ZDEBGtest_utilization (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
4332025-04-30T06:43:41.614ZTRACtest_utilization (DataLoader): authorize begin action = CreateChild actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) }
4342025-04-30T06:43:41.614ZTRACtest_utilization (DataLoader): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. } resource_id = 001de000-4401-4000-8000-000000000000 resource_type = Project
4352025-04-30T06:43:41.614ZTRACtest_utilization (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
4362025-04-30T06:43:41.614ZDEBGtest_utilization (DataLoader): roles roles = RoleSet { roles: {} }
4372025-04-30T06:43:41.615ZDEBGtest_utilization (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
4382025-04-30T06:43:41.619ZTRACtest_utilization (DataLoader): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. } resource_id = 001de000-5110-4000-8000-000000000001 resource_type = Silo
4392025-04-30T06:43:41.619ZTRACtest_utilization (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
4402025-04-30T06:43:41.619ZDEBGtest_utilization (DataLoader): roles roles = RoleSet { roles: {} }
4412025-04-30T06:43:41.620ZDEBGtest_utilization (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
4422025-04-30T06:43:41.629ZTRACtest_utilization (DataLoader): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
4432025-04-30T06:43:41.629ZTRACtest_utilization (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
4442025-04-30T06:43:41.629ZDEBGtest_utilization (DataLoader): roles roles = RoleSet { roles: {} }
4452025-04-30T06:43:41.629ZDEBGtest_utilization (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
4462025-04-30T06:43:41.632ZDEBGtest_utilization (DataLoader): roles roles = RoleSet { roles: {} }
4472025-04-30T06:43:41.646ZDEBGtest_utilization (DataLoader): authorize result action = CreateChild actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
4482025-04-30T06:43:41.647ZTRACtest_utilization (DataLoader): successful lookup of siloed resource "Vpc" using built-in user
4492025-04-30T06:43:41.647ZTRACtest_utilization (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
4502025-04-30T06:43:41.647ZDEBGtest_utilization (DataLoader): roles roles = RoleSet { roles: {} }
4512025-04-30T06:43:41.648ZDEBGtest_utilization (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
4522025-04-30T06:43:41.655ZTRACtest_utilization (DataLoader): authorize begin action = CreateChild actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) }
4532025-04-30T06:43:41.655ZTRACtest_utilization (DataLoader): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. } resource_id = 001de000-4401-4000-8000-000000000000 resource_type = Project
4542025-04-30T06:43:41.655ZTRACtest_utilization (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
4552025-04-30T06:43:41.655ZDEBGtest_utilization (DataLoader): roles roles = RoleSet { roles: {} }
4562025-04-30T06:43:41.656ZDEBGtest_utilization (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
4572025-04-30T06:43:41.659ZTRACtest_utilization (DataLoader): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. } resource_id = 001de000-5110-4000-8000-000000000001 resource_type = Silo
4582025-04-30T06:43:41.659ZTRACtest_utilization (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
4592025-04-30T06:43:41.659ZDEBGtest_utilization (DataLoader): roles roles = RoleSet { roles: {} }
4602025-04-30T06:43:41.660ZDEBGtest_utilization (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
4612025-04-30T06:43:41.664ZTRACtest_utilization (DataLoader): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
4622025-04-30T06:43:41.664ZTRACtest_utilization (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
4632025-04-30T06:43:41.664ZDEBGtest_utilization (DataLoader): roles roles = RoleSet { roles: {} }
4642025-04-30T06:43:41.665ZDEBGtest_utilization (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
4652025-04-30T06:43:41.667ZDEBGtest_utilization (DataLoader): roles roles = RoleSet { roles: {} }
4662025-04-30T06:43:41.675ZDEBGtest_utilization (DataLoader): authorize result action = CreateChild actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
4672025-04-30T06:43:41.681ZTRACtest_utilization (DataLoader): authorize begin action = CreateChild actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = VpcRouter { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-074c-4000-8000-000000000001, lookup_type: ById(001de000-074c-4000-8000-000000000001) }
4682025-04-30T06:43:41.682ZTRACtest_utilization (DataLoader): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. } resource_id = 001de000-4401-4000-8000-000000000000 resource_type = Project
4692025-04-30T06:43:41.682ZTRACtest_utilization (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
4702025-04-30T06:43:41.682ZDEBGtest_utilization (DataLoader): roles roles = RoleSet { roles: {} }
4712025-04-30T06:43:41.684ZDEBGtest_utilization (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
4722025-04-30T06:43:41.686ZTRACtest_utilization (DataLoader): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. } resource_id = 001de000-5110-4000-8000-000000000001 resource_type = Silo
4732025-04-30T06:43:41.686ZTRACtest_utilization (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
4742025-04-30T06:43:41.686ZDEBGtest_utilization (DataLoader): roles roles = RoleSet { roles: {} }
4752025-04-30T06:43:41.687ZDEBGtest_utilization (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
4762025-04-30T06:43:41.689ZTRACtest_utilization (DataLoader): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
4772025-04-30T06:43:41.689ZTRACtest_utilization (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
4782025-04-30T06:43:41.689ZDEBGtest_utilization (DataLoader): roles roles = RoleSet { roles: {} }
4792025-04-30T06:43:41.690ZDEBGtest_utilization (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
4802025-04-30T06:43:41.692ZDEBGtest_utilization (DataLoader): roles roles = RoleSet { roles: {} }
4812025-04-30T06:43:41.701ZDEBGtest_utilization (DataLoader): authorize result action = CreateChild actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = VpcRouter { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-074c-4000-8000-000000000001, lookup_type: ById(001de000-074c-4000-8000-000000000001) } result = Ok(())
4822025-04-30T06:43:41.701ZTRACtest_utilization (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
4832025-04-30T06:43:41.701ZDEBGtest_utilization (DataLoader): roles roles = RoleSet { roles: {} }
4842025-04-30T06:43:41.703ZDEBGtest_utilization (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
4852025-04-30T06:43:41.773ZTRACtest_utilization (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
4862025-04-30T06:43:41.773ZDEBGtest_utilization (DataLoader): roles roles = RoleSet { roles: {} }
4872025-04-30T06:43:41.773ZDEBGtest_utilization (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
4882025-04-30T06:43:41.778ZTRACtest_utilization (DataLoader): authorize begin action = CreateChild actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) }
4892025-04-30T06:43:41.778ZTRACtest_utilization (DataLoader): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. } resource_id = 001de000-4401-4000-8000-000000000000 resource_type = Project
4902025-04-30T06:43:41.778ZTRACtest_utilization (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
4912025-04-30T06:43:41.778ZDEBGtest_utilization (DataLoader): roles roles = RoleSet { roles: {} }
4922025-04-30T06:43:41.779ZDEBGtest_utilization (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
4932025-04-30T06:43:41.782ZTRACtest_utilization (DataLoader): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. } resource_id = 001de000-5110-4000-8000-000000000001 resource_type = Silo
4942025-04-30T06:43:41.782ZTRACtest_utilization (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
4952025-04-30T06:43:41.782ZDEBGtest_utilization (DataLoader): roles roles = RoleSet { roles: {} }
4962025-04-30T06:43:41.783ZDEBGtest_utilization (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
4972025-04-30T06:43:41.787ZTRACtest_utilization (DataLoader): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
4982025-04-30T06:43:41.787ZTRACtest_utilization (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
4992025-04-30T06:43:41.787ZDEBGtest_utilization (DataLoader): roles roles = RoleSet { roles: {} }
5002025-04-30T06:43:41.789ZDEBGtest_utilization (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
5012025-04-30T06:43:41.791ZDEBGtest_utilization (DataLoader): roles roles = RoleSet { roles: {} }
5022025-04-30T06:43:41.810ZDEBGtest_utilization (DataLoader): authorize result action = CreateChild actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
5032025-04-30T06:43:41.817ZTRACtest_utilization (DataLoader): authorize begin action = CreateChild actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = VpcRouter { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-074c-4000-8000-000000000001, lookup_type: ById(001de000-074c-4000-8000-000000000001) }
5042025-04-30T06:43:41.817ZTRACtest_utilization (DataLoader): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. } resource_id = 001de000-4401-4000-8000-000000000000 resource_type = Project
5052025-04-30T06:43:41.817ZTRACtest_utilization (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
5062025-04-30T06:43:41.817ZDEBGtest_utilization (DataLoader): roles roles = RoleSet { roles: {} }
5072025-04-30T06:43:41.821ZDEBGtest_utilization (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
5082025-04-30T06:43:41.823ZTRACtest_utilization (DataLoader): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. } resource_id = 001de000-5110-4000-8000-000000000001 resource_type = Silo
5092025-04-30T06:43:41.824ZTRACtest_utilization (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
5102025-04-30T06:43:41.824ZDEBGtest_utilization (DataLoader): roles roles = RoleSet { roles: {} }
5112025-04-30T06:43:41.824ZDEBGtest_utilization (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
5122025-04-30T06:43:41.828ZTRACtest_utilization (DataLoader): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
5132025-04-30T06:43:41.828ZTRACtest_utilization (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
5142025-04-30T06:43:41.828ZDEBGtest_utilization (DataLoader): roles roles = RoleSet { roles: {} }
5152025-04-30T06:43:41.829ZDEBGtest_utilization (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
5162025-04-30T06:43:41.831ZDEBGtest_utilization (DataLoader): roles roles = RoleSet { roles: {} }
5172025-04-30T06:43:41.840ZDEBGtest_utilization (DataLoader): authorize result action = CreateChild actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = VpcRouter { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-074c-4000-8000-000000000001, lookup_type: ById(001de000-074c-4000-8000-000000000001) } result = Ok(())
5182025-04-30T06:43:41.840ZTRACtest_utilization (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
5192025-04-30T06:43:41.840ZDEBGtest_utilization (DataLoader): roles roles = RoleSet { roles: {} }
5202025-04-30T06:43:41.841ZDEBGtest_utilization (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
5212025-04-30T06:43:41.847ZTRACtest_utilization (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
5222025-04-30T06:43:41.847ZDEBGtest_utilization (DataLoader): roles roles = RoleSet { roles: {} }
5232025-04-30T06:43:41.848ZDEBGtest_utilization (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
5242025-04-30T06:43:41.851ZTRACtest_utilization (DataLoader): authorize begin action = CreateChild actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) }
5252025-04-30T06:43:41.851ZTRACtest_utilization (DataLoader): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. } resource_id = 001de000-4401-4000-8000-000000000000 resource_type = Project
5262025-04-30T06:43:41.851ZTRACtest_utilization (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
5272025-04-30T06:43:41.851ZDEBGtest_utilization (DataLoader): roles roles = RoleSet { roles: {} }
5282025-04-30T06:43:41.851ZDEBGtest_utilization (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
5292025-04-30T06:43:41.853ZTRACtest_utilization (DataLoader): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. } resource_id = 001de000-5110-4000-8000-000000000001 resource_type = Silo
5302025-04-30T06:43:41.853ZTRACtest_utilization (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
5312025-04-30T06:43:41.853ZDEBGtest_utilization (DataLoader): roles roles = RoleSet { roles: {} }
5322025-04-30T06:43:41.854ZDEBGtest_utilization (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
5332025-04-30T06:43:41.856ZTRACtest_utilization (DataLoader): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
5342025-04-30T06:43:41.856ZTRACtest_utilization (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
5352025-04-30T06:43:41.856ZDEBGtest_utilization (DataLoader): roles roles = RoleSet { roles: {} }
5362025-04-30T06:43:41.857ZDEBGtest_utilization (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
5372025-04-30T06:43:41.859ZDEBGtest_utilization (DataLoader): roles roles = RoleSet { roles: {} }
5382025-04-30T06:43:41.866ZDEBGtest_utilization (DataLoader): authorize result action = CreateChild actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
5392025-04-30T06:43:41.871ZTRACtest_utilization (DataLoader): authorize begin action = CreateChild actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = VpcRouter { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-074c-4000-8000-000000000001, lookup_type: ById(001de000-074c-4000-8000-000000000001) }
5402025-04-30T06:43:41.871ZTRACtest_utilization (DataLoader): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. } resource_id = 001de000-4401-4000-8000-000000000000 resource_type = Project
5412025-04-30T06:43:41.871ZTRACtest_utilization (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
5422025-04-30T06:43:41.871ZDEBGtest_utilization (DataLoader): roles roles = RoleSet { roles: {} }
5432025-04-30T06:43:41.871ZDEBGtest_utilization (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
5442025-04-30T06:43:41.873ZTRACtest_utilization (DataLoader): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. } resource_id = 001de000-5110-4000-8000-000000000001 resource_type = Silo
5452025-04-30T06:43:41.874ZTRACtest_utilization (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
5462025-04-30T06:43:41.874ZDEBGtest_utilization (DataLoader): roles roles = RoleSet { roles: {} }
5472025-04-30T06:43:41.874ZDEBGtest_utilization (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
5482025-04-30T06:43:41.877ZTRACtest_utilization (DataLoader): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
5492025-04-30T06:43:41.877ZTRACtest_utilization (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
5502025-04-30T06:43:41.877ZDEBGtest_utilization (DataLoader): roles roles = RoleSet { roles: {} }
5512025-04-30T06:43:41.878ZDEBGtest_utilization (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
5522025-04-30T06:43:41.881ZDEBGtest_utilization (DataLoader): roles roles = RoleSet { roles: {} }
5532025-04-30T06:43:41.890ZDEBGtest_utilization (DataLoader): authorize result action = CreateChild actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = VpcRouter { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-074c-4000-8000-000000000001, lookup_type: ById(001de000-074c-4000-8000-000000000001) } result = Ok(())
5542025-04-30T06:43:41.890ZTRACtest_utilization (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
5552025-04-30T06:43:41.890ZDEBGtest_utilization (DataLoader): roles roles = RoleSet { roles: {} }
5562025-04-30T06:43:41.890ZDEBGtest_utilization (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
5572025-04-30T06:43:41.897ZTRACtest_utilization (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
5582025-04-30T06:43:41.897ZDEBGtest_utilization (DataLoader): roles roles = RoleSet { roles: {} }
5592025-04-30T06:43:41.899ZDEBGtest_utilization (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
5602025-04-30T06:43:41.903ZINFOtest_utilization (DataLoader): created built-in services vpc subnets
5612025-04-30T06:43:41.903ZINFOtest_utilization (DataLoader): created built-in services vpc
5622025-04-30T06:43:41.903ZTRACtest_utilization (DataLoader): authorize begin action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
5632025-04-30T06:43:41.903ZDEBGtest_utilization (DataLoader): roles roles = RoleSet { roles: {} }
5642025-04-30T06:43:41.905ZDEBGtest_utilization (DataLoader): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
5652025-04-30T06:43:41.905ZDEBGtest_utilization (DataLoader): attempting to create silo users
5662025-04-30T06:43:41.905ZTRACtest_utilization (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
5672025-04-30T06:43:41.905ZDEBGtest_utilization (DataLoader): roles roles = RoleSet { roles: {} }
5682025-04-30T06:43:41.906ZDEBGtest_utilization (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
5692025-04-30T06:43:41.913ZINFOtest_utilization (DataLoader): created 2 silo users
5702025-04-30T06:43:41.914ZDEBGtest_utilization (DataLoader): attempting to create silo user role assignments
5712025-04-30T06:43:41.914ZTRACtest_utilization (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
5722025-04-30T06:43:41.914ZDEBGtest_utilization (DataLoader): roles roles = RoleSet { roles: {} }
5732025-04-30T06:43:41.917ZDEBGtest_utilization (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
5742025-04-30T06:43:41.921ZINFOtest_utilization (DataLoader): created 2 silo user role assignments
5752025-04-30T06:43:41.921ZTRACtest_utilization (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
5762025-04-30T06:43:41.921ZDEBGtest_utilization (DataLoader): roles roles = RoleSet { roles: {} }
5772025-04-30T06:43:41.922ZDEBGtest_utilization (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
5782025-04-30T06:43:41.931ZTRACtest_utilization (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
5792025-04-30T06:43:41.931ZDEBGtest_utilization (DataLoader): roles roles = RoleSet { roles: {} }
5802025-04-30T06:43:41.932ZDEBGtest_utilization (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
5812025-04-30T06:43:41.941ZTRACtest_utilization (DataLoader): authorize begin action = CreateChild actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = IpPoolList
5822025-04-30T06:43:41.941ZTRACtest_utilization (DataLoader): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
5832025-04-30T06:43:41.941ZTRACtest_utilization (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
5842025-04-30T06:43:41.941ZDEBGtest_utilization (DataLoader): roles roles = RoleSet { roles: {} }
5852025-04-30T06:43:41.942ZDEBGtest_utilization (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
5862025-04-30T06:43:41.945ZDEBGtest_utilization (DataLoader): roles roles = RoleSet { roles: {} }
5872025-04-30T06:43:41.947ZDEBGtest_utilization (DataLoader): authorize result action = CreateChild actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = IpPoolList result = Ok(())
5882025-04-30T06:43:41.947ZTRACtest_utilization (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
5892025-04-30T06:43:41.947ZDEBGtest_utilization (DataLoader): roles roles = RoleSet { roles: {} }
5902025-04-30T06:43:41.948ZDEBGtest_utilization (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
5912025-04-30T06:43:41.955ZTRACtest_utilization (DataLoader): authorize begin action = CreateChild actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = IpPoolList
5922025-04-30T06:43:41.955ZTRACtest_utilization (DataLoader): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
5932025-04-30T06:43:41.955ZTRACtest_utilization (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
5942025-04-30T06:43:41.955ZDEBGtest_utilization (DataLoader): roles roles = RoleSet { roles: {} }
5952025-04-30T06:43:41.956ZDEBGtest_utilization (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
5962025-04-30T06:43:41.959ZDEBGtest_utilization (DataLoader): roles roles = RoleSet { roles: {} }
5972025-04-30T06:43:41.963ZDEBGtest_utilization (DataLoader): authorize result action = CreateChild actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = IpPoolList result = Ok(())
5982025-04-30T06:43:41.963ZTRACtest_utilization (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
5992025-04-30T06:43:41.963ZDEBGtest_utilization (DataLoader): roles roles = RoleSet { roles: {} }
6002025-04-30T06:43:41.964ZDEBGtest_utilization (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
6012025-04-30T06:43:42.073ZTRACtest_utilization (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
6022025-04-30T06:43:42.074ZDEBGtest_utilization (DataLoader): roles roles = RoleSet { roles: {} }
6032025-04-30T06:43:42.085ZDEBGtest_utilization (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
6042025-04-30T06:43:42.134ZINFOtest_utilization (nexus): populate complete
6052025-04-30T06:43:42.135ZINFOtest_utilization (nexus): activating background tasks
6062025-04-30T06:43:42.139ZDEBGtest_utilization (init_with_steps): Running step start_sled1
6072025-04-30T06:43:42.139ZINFOtest_utilization (omicron_sled_agent::sim::Server): setting up sled agent server sled_id = b6d65341-167c-41df-9b5c-41cded99c229
6082025-04-30T06:43:42.220ZINFOtest_utilization (SledAgent): created simulated sled agent server = b6d65341-167c-41df-9b5c-41cded99c229 sim_mode = Explicit sled_id = b6d65341-167c-41df-9b5c-41cded99c229
6092025-04-30T06:43:42.220ZINFOtest_utilization (omicron_sled_agent::sim::SimulatedUpstairs): registered sled b6d65341-167c-41df-9b5c-41cded99c229 storage
6102025-04-30T06:43:42.222ZDEBGtest_utilization (BackgroundTasks): activating background_task = dns_config_internal dns_group = internal iteration = 1 reason = Timeout
6112025-04-30T06:43:42.222ZDEBGtest_utilization (BackgroundTasks): reading DNS version background_task = dns_config_internal dns_group = internal
6122025-04-30T06:43:42.222ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database
6132025-04-30T06:43:42.222ZDEBGtest_utilization (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
6142025-04-30T06:43:42.222ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
6152025-04-30T06:43:42.223ZDEBGtest_utilization (BackgroundTasks): activating background_task = dns_servers_internal dns_group = internal iteration = 1 reason = Timeout
6162025-04-30T06:43:42.223ZTRACtest_utilization (DnsResolver): lookup_all_socket_v6 srv dns_name = _nameservice._tcp.control-plane.oxide.internal
6172025-04-30T06:43:42.223ZDEBGtest_utilization (BackgroundTasks): activating background_task = dns_propagation_internal dns_group = internal iteration = 1 reason = Timeout
6182025-04-30T06:43:42.223ZWARNtest_utilization (BackgroundTasks): DNS propagation: skipped background_task = dns_propagation_internal dns_group = internal reason = no config nor servers
6192025-04-30T06:43:42.223ZDEBGtest_utilization (BackgroundTasks): activation complete background_task = dns_propagation_internal dns_group = internal elapsed = 107.364\u{b5}s iteration = 1 status = {"error":"no config nor servers"}
6202025-04-30T06:43:42.224ZDEBGtest_utilization (BackgroundTasks): activating background_task = dns_config_external dns_group = external iteration = 1 reason = Timeout
6212025-04-30T06:43:42.224ZDEBGtest_utilization (BackgroundTasks): reading DNS version background_task = dns_config_external dns_group = external
6222025-04-30T06:43:42.224ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database
6232025-04-30T06:43:42.224ZDEBGtest_utilization (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
6242025-04-30T06:43:42.225ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
6252025-04-30T06:43:42.225ZDEBGtest_utilization (BackgroundTasks): activating background_task = dns_servers_external dns_group = external iteration = 1 reason = Timeout
6262025-04-30T06:43:42.225ZTRACtest_utilization (DnsResolver): lookup_all_socket_v6 srv dns_name = _external-dns._tcp.control-plane.oxide.internal
6272025-04-30T06:43:42.225ZDEBGtest_utilization (BackgroundTasks): activating background_task = dns_propagation_external dns_group = external iteration = 1 reason = Timeout
6282025-04-30T06:43:42.225ZWARNtest_utilization (BackgroundTasks): DNS propagation: skipped background_task = dns_propagation_external dns_group = external reason = no config nor servers
6292025-04-30T06:43:42.225ZDEBGtest_utilization (BackgroundTasks): activation complete background_task = dns_propagation_external dns_group = external elapsed = 47.535\u{b5}s iteration = 1 status = {"error":"no config nor servers"}
6302025-04-30T06:43:42.225ZDEBGtest_utilization (BackgroundTasks): activating background_task = metrics_producer_gc iteration = 1 reason = Timeout
6312025-04-30T06:43:42.225ZINFOtest_utilization (BackgroundTasks): Metric producer GC running background_task = metrics_producer_gc expiration = 2025-04-30 06:33:42.225433951 UTC
6322025-04-30T06:43:42.225ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = metrics_producer_gc resource = Database
6332025-04-30T06:43:42.225ZDEBGtest_utilization (BackgroundTasks): roles background_task = metrics_producer_gc roles = RoleSet { roles: {} }
6342025-04-30T06:43:42.226ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = metrics_producer_gc resource = Database result = Ok(())
6352025-04-30T06:43:42.226ZDEBGtest_utilization (BackgroundTasks): activating background_task = external_endpoints iteration = 1 reason = Timeout
6362025-04-30T06:43:42.226ZTRACtest_utilization (BackgroundTasks): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet
6372025-04-30T06:43:42.226ZTRACtest_utilization (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = external_endpoints resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
6382025-04-30T06:43:42.226ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database
6392025-04-30T06:43:42.226ZDEBGtest_utilization (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
6402025-04-30T06:43:42.226ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
6412025-04-30T06:43:42.226ZDEBGtest_utilization (BackgroundTasks): activating background_task = nat_v4_garbage_collector iteration = 1 reason = Timeout
6422025-04-30T06:43:42.226ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database
6432025-04-30T06:43:42.226ZDEBGtest_utilization (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
6442025-04-30T06:43:42.231ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
6452025-04-30T06:43:42.231ZDEBGtest_utilization (BackgroundTasks): activating background_task = bfd_manager iteration = 1 reason = Timeout
6462025-04-30T06:43:42.231ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database
6472025-04-30T06:43:42.231ZDEBGtest_utilization (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
6482025-04-30T06:43:42.232ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
6492025-04-30T06:43:42.232ZDEBGtest_utilization (BackgroundTasks): activating background_task = phantom_disks iteration = 1 reason = Timeout
6502025-04-30T06:43:42.232ZDEBGtest_utilization (BackgroundTasks): activating background_task = blueprint_loader iteration = 1 reason = Timeout
6512025-04-30T06:43:42.232ZTRACtest_utilization (BackgroundTasks): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig
6522025-04-30T06:43:42.232ZTRACtest_utilization (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = blueprint_loader resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
6532025-04-30T06:43:42.232ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database
6542025-04-30T06:43:42.232ZDEBGtest_utilization (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
6552025-04-30T06:43:42.232ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
6562025-04-30T06:43:42.232ZDEBGtest_utilization (BackgroundTasks): activating background_task = blueprint_executor iteration = 1 reason = Timeout
6572025-04-30T06:43:42.232ZWARNtest_utilization (BackgroundTasks): Blueprint execution: skipped background_task = blueprint_executor reason = no blueprint
6582025-04-30T06:43:42.232ZDEBGtest_utilization (BackgroundTasks): activation complete background_task = blueprint_executor elapsed = 72.865\u{b5}s iteration = 1 status = {"error":"no blueprint"}
6592025-04-30T06:43:42.233ZDEBGtest_utilization (BackgroundTasks): activating background_task = crdb_node_id_collector iteration = 1 reason = Timeout
6602025-04-30T06:43:42.233ZWARNtest_utilization (BackgroundTasks): Blueprint execution: skipped background_task = crdb_node_id_collector reason = no blueprint
6612025-04-30T06:43:42.233ZDEBGtest_utilization (BackgroundTasks): activation complete background_task = crdb_node_id_collector elapsed = 36.903\u{b5}s iteration = 1 status = {"error":"no blueprint"}
6622025-04-30T06:43:42.233ZDEBGtest_utilization (BackgroundTasks): activating background_task = inventory_collection iteration = 1 reason = Timeout
6632025-04-30T06:43:42.233ZTRACtest_utilization (BackgroundTasks): authorize begin action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory
6642025-04-30T06:43:42.233ZTRACtest_utilization (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
6652025-04-30T06:43:42.233ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
6662025-04-30T06:43:42.233ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
6672025-04-30T06:43:42.233ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
6682025-04-30T06:43:42.233ZDEBGtest_utilization (BackgroundTasks): activating background_task = support_bundle_collector iteration = 1 reason = Timeout
6692025-04-30T06:43:42.233ZTRACtest_utilization (BackgroundTasks): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Fleet
6702025-04-30T06:43:42.233ZTRACtest_utilization (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = support_bundle_collector resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
6712025-04-30T06:43:42.233ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database
6722025-04-30T06:43:42.233ZDEBGtest_utilization (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
6732025-04-30T06:43:42.234ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
6742025-04-30T06:43:42.234ZDEBGtest_utilization (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 1 reason = Timeout
6752025-04-30T06:43:42.234ZDEBGtest_utilization (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 21.513\u{b5}s iteration = 1 status = {"error":"task disabled"}
6762025-04-30T06:43:42.234ZDEBGtest_utilization (BackgroundTasks): activating background_task = blueprint_rendezvous iteration = 1 reason = Timeout
6772025-04-30T06:43:42.234ZWARNtest_utilization (BackgroundTasks): Blueprint rendezvous: skipped background_task = blueprint_rendezvous reason = no blueprint
6782025-04-30T06:43:42.234ZDEBGtest_utilization (BackgroundTasks): activation complete background_task = blueprint_rendezvous elapsed = 22.044\u{b5}s iteration = 1 status = {"error":"no blueprint"}
6792025-04-30T06:43:42.234ZDEBGtest_utilization (BackgroundTasks): activating background_task = decommissioned_disk_cleaner iteration = 1 reason = Timeout
6802025-04-30T06:43:42.234ZINFOtest_utilization (BackgroundTasks): Decommissioned Disk Cleaner disabled background_task = decommissioned_disk_cleaner
6812025-04-30T06:43:42.234ZDEBGtest_utilization (BackgroundTasks): activation complete background_task = decommissioned_disk_cleaner elapsed = 82.695\u{b5}s iteration = 1 status = {"deleted":0,"error":null,"error_count":0,"found":0,"not_ready_to_be_deleted":0}
6822025-04-30T06:43:42.234ZDEBGtest_utilization (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 1 reason = Timeout
6832025-04-30T06:43:42.234ZTRACtest_utilization (BackgroundTasks): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory
6842025-04-30T06:43:42.234ZTRACtest_utilization (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = service_zone_nat_tracker resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
6852025-04-30T06:43:42.234ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database
6862025-04-30T06:43:42.234ZDEBGtest_utilization (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
6872025-04-30T06:43:42.235ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
6882025-04-30T06:43:42.235ZDEBGtest_utilization (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 1 reason = Timeout
6892025-04-30T06:43:42.235ZTRACtest_utilization (BackgroundTasks): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Fleet
6902025-04-30T06:43:42.235ZTRACtest_utilization (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = switch_port_config_manager resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
6912025-04-30T06:43:42.235ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database
6922025-04-30T06:43:42.235ZDEBGtest_utilization (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
6932025-04-30T06:43:42.235ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
6942025-04-30T06:43:42.236ZDEBGtest_utilization (BackgroundTasks): activating background_task = v2p_manager iteration = 1 reason = Timeout
6952025-04-30T06:43:42.236ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database
6962025-04-30T06:43:42.236ZDEBGtest_utilization (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
6972025-04-30T06:43:42.237ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
6982025-04-30T06:43:42.237ZDEBGtest_utilization (BackgroundTasks): activating background_task = region_replacement iteration = 1 reason = Timeout
6992025-04-30T06:43:42.237ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database
7002025-04-30T06:43:42.237ZDEBGtest_utilization (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
7012025-04-30T06:43:42.237ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
7022025-04-30T06:43:42.237ZDEBGtest_utilization (BackgroundTasks): activating background_task = region_replacement_driver iteration = 1 reason = Timeout
7032025-04-30T06:43:42.237ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database
7042025-04-30T06:43:42.237ZDEBGtest_utilization (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
7052025-04-30T06:43:42.238ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
7062025-04-30T06:43:42.238ZDEBGtest_utilization (BackgroundTasks): activating background_task = instance_watcher iteration = 1 reason = Timeout
7072025-04-30T06:43:42.238ZTRACtest_utilization (BackgroundTasks): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Fleet
7082025-04-30T06:43:42.238ZTRACtest_utilization (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = instance_watcher resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
7092025-04-30T06:43:42.238ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database
7102025-04-30T06:43:42.238ZDEBGtest_utilization (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
7112025-04-30T06:43:42.239ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
7122025-04-30T06:43:42.239ZDEBGtest_utilization (BackgroundTasks): activating background_task = instance_updater iteration = 1 reason = Timeout
7132025-04-30T06:43:42.239ZINFOtest_utilization (BackgroundTasks): background instance updater explicitly disabled background_task = instance_updater
7142025-04-30T06:43:42.240ZDEBGtest_utilization (BackgroundTasks): activation complete background_task = instance_updater elapsed = 1.016555ms iteration = 1 status = {"destroyed_active_vmms":0,"disabled":true,"failed_active_vmms":0,"query_errors":[],"saga_errors":[],"sagas_completed":0,"sagas_started":0,"terminated_active_migrations":0}
7152025-04-30T06:43:42.240ZDEBGtest_utilization (BackgroundTasks): activating background_task = instance_reincarnation iteration = 1 reason = Timeout
7162025-04-30T06:43:42.240ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_reincarnation resource = Database
7172025-04-30T06:43:42.240ZDEBGtest_utilization (BackgroundTasks): roles background_task = instance_reincarnation roles = RoleSet { roles: {} }
7182025-04-30T06:43:42.242ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_reincarnation resource = Database result = Ok(())
7192025-04-30T06:43:42.242ZDEBGtest_utilization (BackgroundTasks): activating background_task = service_firewall_rule_propagation iteration = 1 reason = Timeout
7202025-04-30T06:43:42.242ZDEBGtest_utilization (service-firewall-rule-progator): starting background task for service firewall rule propagation background_task = service_firewall_rule_propagation
7212025-04-30T06:43:42.242ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database
7222025-04-30T06:43:42.242ZDEBGtest_utilization (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
7232025-04-30T06:43:42.242ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
7242025-04-30T06:43:42.242ZDEBGtest_utilization (BackgroundTasks): activating background_task = vpc_route_manager iteration = 1 reason = Timeout
7252025-04-30T06:43:42.242ZINFOtest_utilization (BackgroundTasks): VPC route manager running background_task = vpc_route_manager
7262025-04-30T06:43:42.242ZTRACtest_utilization (BackgroundTasks): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet
7272025-04-30T06:43:42.243ZTRACtest_utilization (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = vpc_route_manager resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
7282025-04-30T06:43:42.243ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database
7292025-04-30T06:43:42.243ZDEBGtest_utilization (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
7302025-04-30T06:43:42.243ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
7312025-04-30T06:43:42.243ZDEBGtest_utilization (BackgroundTasks): activating background_task = abandoned_vmm_reaper iteration = 1 reason = Timeout
7322025-04-30T06:43:42.243ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = abandoned_vmm_reaper resource = Database
7332025-04-30T06:43:42.243ZDEBGtest_utilization (BackgroundTasks): roles background_task = abandoned_vmm_reaper roles = RoleSet { roles: {} }
7342025-04-30T06:43:42.244ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = abandoned_vmm_reaper resource = Database result = Ok(())
7352025-04-30T06:43:42.244ZDEBGtest_utilization (BackgroundTasks): activating background_task = saga_recovery iteration = 1 reason = Timeout
7362025-04-30T06:43:42.244ZDEBGtest_utilization (SagaRecoverer): listing candidate sagas for recovery
7372025-04-30T06:43:42.244ZTRACtest_utilization (SagaRecoverer): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000003a8a, .. }) resource = Database
7382025-04-30T06:43:42.244ZDEBGtest_utilization (SagaRecoverer): roles roles = RoleSet { roles: {} }
7392025-04-30T06:43:42.244ZDEBGtest_utilization (SagaRecoverer): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000003a8a, .. }) resource = Database result = Ok(())
7402025-04-30T06:43:42.244ZDEBGtest_utilization (BackgroundTasks): activating background_task = lookup_region_port iteration = 1 reason = Timeout
7412025-04-30T06:43:42.244ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = lookup_region_port resource = Database
7422025-04-30T06:43:42.244ZDEBGtest_utilization (BackgroundTasks): roles background_task = lookup_region_port roles = RoleSet { roles: {} }
7432025-04-30T06:43:42.245ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = lookup_region_port resource = Database result = Ok(())
7442025-04-30T06:43:42.245ZDEBGtest_utilization (BackgroundTasks): activating background_task = region_snapshot_replacement_start iteration = 1 reason = Timeout
7452025-04-30T06:43:42.245ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database
7462025-04-30T06:43:42.245ZDEBGtest_utilization (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
7472025-04-30T06:43:42.245ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
7482025-04-30T06:43:42.245ZDEBGtest_utilization (BackgroundTasks): activating background_task = region_snapshot_replacement_garbage_collection iteration = 1 reason = Timeout
7492025-04-30T06:43:42.245ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_garbage_collection resource = Database
7502025-04-30T06:43:42.245ZDEBGtest_utilization (BackgroundTasks): roles background_task = region_snapshot_replacement_garbage_collection roles = RoleSet { roles: {} }
7512025-04-30T06:43:42.248ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_garbage_collection resource = Database result = Ok(())
7522025-04-30T06:43:42.248ZDEBGtest_utilization (BackgroundTasks): activating background_task = region_snapshot_replacement_step iteration = 1 reason = Timeout
7532025-04-30T06:43:42.248ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database
7542025-04-30T06:43:42.248ZDEBGtest_utilization (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
7552025-04-30T06:43:42.249ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
7562025-04-30T06:43:42.249ZDEBGtest_utilization (BackgroundTasks): activating background_task = region_snapshot_replacement_finish iteration = 1 reason = Timeout
7572025-04-30T06:43:42.249ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_finish resource = Database
7582025-04-30T06:43:42.249ZDEBGtest_utilization (BackgroundTasks): roles background_task = region_snapshot_replacement_finish roles = RoleSet { roles: {} }
7592025-04-30T06:43:42.249ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_finish resource = Database result = Ok(())
7602025-04-30T06:43:42.249ZDEBGtest_utilization (BackgroundTasks): activating background_task = tuf_artifact_replication iteration = 1 reason = Timeout
7612025-04-30T06:43:42.250ZTRACtest_utilization (BackgroundTasks): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = tuf_artifact_replication resource = Fleet
7622025-04-30T06:43:42.250ZTRACtest_utilization (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = tuf_artifact_replication resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
7632025-04-30T06:43:42.250ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = tuf_artifact_replication resource = Database
7642025-04-30T06:43:42.250ZDEBGtest_utilization (BackgroundTasks): roles background_task = tuf_artifact_replication roles = RoleSet { roles: {} }
7652025-04-30T06:43:42.250ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = tuf_artifact_replication resource = Database result = Ok(())
7662025-04-30T06:43:42.250ZDEBGtest_utilization (BackgroundTasks): activating background_task = read_only_region_replacement_start iteration = 1 reason = Timeout
7672025-04-30T06:43:42.250ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = read_only_region_replacement_start resource = Database
7682025-04-30T06:43:42.250ZDEBGtest_utilization (BackgroundTasks): roles background_task = read_only_region_replacement_start roles = RoleSet { roles: {} }
7692025-04-30T06:43:42.250ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = read_only_region_replacement_start resource = Database result = Ok(())
7702025-04-30T06:43:42.251ZDEBGtest_utilization (BackgroundTasks): activating background_task = webhook_dispatcher iteration = 1 reason = Timeout
7712025-04-30T06:43:42.251ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = webhook_dispatcher resource = Database
7722025-04-30T06:43:42.251ZDEBGtest_utilization (BackgroundTasks): roles background_task = webhook_dispatcher roles = RoleSet { roles: {} }
7732025-04-30T06:43:42.251ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = webhook_dispatcher resource = Database result = Ok(())
7742025-04-30T06:43:42.251ZDEBGtest_utilization (BackgroundTasks): activating background_task = webhook_deliverator iteration = 1 reason = Timeout
7752025-04-30T06:43:42.251ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = webhook_deliverator resource = Database
7762025-04-30T06:43:42.251ZDEBGtest_utilization (BackgroundTasks): roles background_task = webhook_deliverator roles = RoleSet { roles: {} }
7772025-04-30T06:43:42.252ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = webhook_deliverator resource = Database result = Ok(())
7782025-04-30T06:43:42.252ZTRACtest_utilization (BackgroundTasks): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig
7792025-04-30T06:43:42.252ZTRACtest_utilization (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = dns_config_internal dns_group = internal resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
7802025-04-30T06:43:42.252ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database
7812025-04-30T06:43:42.252ZDEBGtest_utilization (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
7822025-04-30T06:43:42.252ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
7832025-04-30T06:43:42.253ZINFOtest_utilization (ArtifactStore): No ledger in /var/tmp/omicron_tmp/.tmp5QxCob/artifact-config.json server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
7842025-04-30T06:43:42.253ZDEBGtest_utilization (ArtifactStore): Failed to read ledger: Not found in storage path = /var/tmp/omicron_tmp/.tmp5QxCob/artifact-config.json server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
7852025-04-30T06:43:42.253ZINFOtest_utilization (ArtifactStore): No ledger in /var/tmp/omicron_tmp/.tmpRU082q/artifact-config.json server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
7862025-04-30T06:43:42.253ZDEBGtest_utilization (ArtifactStore): Failed to read ledger: Not found in storage path = /var/tmp/omicron_tmp/.tmpRU082q/artifact-config.json server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
7872025-04-30T06:43:42.303ZDEBGtest_utilization (dropshot (Repo Depot)): registered endpoint local_addr = [::1]:51171 method = GET path = /artifact/sha256/{sha256} server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
7882025-04-30T06:43:42.304ZINFOtest_utilization (dropshot (Repo Depot)): listening local_addr = [::1]:51171 server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
7892025-04-30T06:43:42.305ZDEBGtest_utilization (dropshot (Repo Depot)): successfully registered DTrace USDT probes local_addr = [::1]:51171 server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
7902025-04-30T06:43:42.312ZDEBGtest_utilization (dropshot): registered endpoint local_addr = [::1]:39365 method = GET path = /artifacts sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
7912025-04-30T06:43:42.312ZDEBGtest_utilization (dropshot): registered endpoint local_addr = [::1]:39365 method = PUT path = /artifacts/{sha256} sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
7922025-04-30T06:43:42.312ZDEBGtest_utilization (dropshot): registered endpoint local_addr = [::1]:39365 method = POST path = /artifacts/{sha256}/copy-from-depot sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
7932025-04-30T06:43:42.312ZDEBGtest_utilization (dropshot): registered endpoint local_addr = [::1]:39365 method = GET path = /artifacts-config sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
7942025-04-30T06:43:42.312ZDEBGtest_utilization (dropshot): registered endpoint local_addr = [::1]:39365 method = PUT path = /artifacts-config sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
7952025-04-30T06:43:42.312ZDEBGtest_utilization (dropshot): registered endpoint local_addr = [::1]:39365 method = POST path = /boot-disk/{boot_disk}/os/write sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
7962025-04-30T06:43:42.312ZDEBGtest_utilization (dropshot): registered endpoint local_addr = [::1]:39365 method = GET path = /boot-disk/{boot_disk}/os/write/status sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
7972025-04-30T06:43:42.312ZDEBGtest_utilization (dropshot): registered endpoint local_addr = [::1]:39365 method = DELETE path = /boot-disk/{boot_disk}/os/write/status/{update_id} sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
7982025-04-30T06:43:42.312ZDEBGtest_utilization (dropshot): registered endpoint local_addr = [::1]:39365 method = GET path = /bootstore/status sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
7992025-04-30T06:43:42.312ZDEBGtest_utilization (dropshot): registered endpoint local_addr = [::1]:39365 method = GET path = /datasets sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
8002025-04-30T06:43:42.312ZDEBGtest_utilization (dropshot): registered endpoint local_addr = [::1]:39365 method = PUT path = /disks/{disk_id} sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
8012025-04-30T06:43:42.312ZDEBGtest_utilization (dropshot): registered endpoint local_addr = [::1]:39365 method = POST path = /disks/{disk_id}/poke sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
8022025-04-30T06:43:42.312ZDEBGtest_utilization (dropshot): registered endpoint local_addr = [::1]:39365 method = PUT path = /eip-gateways sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
8032025-04-30T06:43:42.312ZDEBGtest_utilization (dropshot): registered endpoint local_addr = [::1]:39365 method = GET path = /inventory sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
8042025-04-30T06:43:42.312ZDEBGtest_utilization (dropshot): registered endpoint local_addr = [::1]:39365 method = GET path = /network-bootstore-config sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
8052025-04-30T06:43:42.312ZDEBGtest_utilization (dropshot): registered endpoint local_addr = [::1]:39365 method = PUT path = /network-bootstore-config sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
8062025-04-30T06:43:42.312ZDEBGtest_utilization (dropshot): registered endpoint local_addr = [::1]:39365 method = PUT path = /omicron-config sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
8072025-04-30T06:43:42.312ZDEBGtest_utilization (dropshot): registered endpoint local_addr = [::1]:39365 method = GET path = /omicron-physical-disks sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
8082025-04-30T06:43:42.312ZDEBGtest_utilization (dropshot): registered endpoint local_addr = [::1]:39365 method = GET path = /sled-identifiers sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
8092025-04-30T06:43:42.312ZDEBGtest_utilization (dropshot): registered endpoint local_addr = [::1]:39365 method = GET path = /sled-role sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
8102025-04-30T06:43:42.312ZDEBGtest_utilization (dropshot): registered endpoint local_addr = [::1]:39365 method = PUT path = /sleds sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
8112025-04-30T06:43:42.312ZDEBGtest_utilization (dropshot): registered endpoint local_addr = [::1]:39365 method = GET path = /support/dladm-info sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
8122025-04-30T06:43:42.312ZDEBGtest_utilization (dropshot): registered endpoint local_addr = [::1]:39365 method = GET path = /support/ipadm-info sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
8132025-04-30T06:43:42.312ZDEBGtest_utilization (dropshot): registered endpoint local_addr = [::1]:39365 method = GET path = /support/logs/download/{zone} sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
8142025-04-30T06:43:42.312ZDEBGtest_utilization (dropshot): registered endpoint local_addr = [::1]:39365 method = GET path = /support/logs/zones sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
8152025-04-30T06:43:42.312ZDEBGtest_utilization (dropshot): registered endpoint local_addr = [::1]:39365 method = GET path = /support/nvmeadm-info sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
8162025-04-30T06:43:42.312ZDEBGtest_utilization (dropshot): registered endpoint local_addr = [::1]:39365 method = GET path = /support/pargs-info sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
8172025-04-30T06:43:42.312ZDEBGtest_utilization (dropshot): registered endpoint local_addr = [::1]:39365 method = GET path = /support/pfiles-info sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
8182025-04-30T06:43:42.312ZDEBGtest_utilization (dropshot): registered endpoint local_addr = [::1]:39365 method = GET path = /support/pstack-info sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
8192025-04-30T06:43:42.312ZDEBGtest_utilization (dropshot): registered endpoint local_addr = [::1]:39365 method = GET path = /support/zfs-info sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
8202025-04-30T06:43:42.312ZDEBGtest_utilization (dropshot): registered endpoint local_addr = [::1]:39365 method = GET path = /support/zoneadm-info sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
8212025-04-30T06:43:42.312ZDEBGtest_utilization (dropshot): registered endpoint local_addr = [::1]:39365 method = GET path = /support/zpool-info sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
8222025-04-30T06:43:42.312ZDEBGtest_utilization (dropshot): registered endpoint local_addr = [::1]:39365 method = GET path = /support-bundles/{zpool_id}/{dataset_id} sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
8232025-04-30T06:43:42.312ZDEBGtest_utilization (dropshot): registered endpoint local_addr = [::1]:39365 method = DELETE path = /support-bundles/{zpool_id}/{dataset_id}/{support_bundle_id} sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
8242025-04-30T06:43:42.313ZDEBGtest_utilization (dropshot): registered endpoint local_addr = [::1]:39365 method = POST path = /support-bundles/{zpool_id}/{dataset_id}/{support_bundle_id} sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
8252025-04-30T06:43:42.313ZDEBGtest_utilization (dropshot): registered endpoint local_addr = [::1]:39365 method = GET path = /support-bundles/{zpool_id}/{dataset_id}/{support_bundle_id}/download sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
8262025-04-30T06:43:42.313ZDEBGtest_utilization (dropshot): registered endpoint local_addr = [::1]:39365 method = HEAD path = /support-bundles/{zpool_id}/{dataset_id}/{support_bundle_id}/download sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
8272025-04-30T06:43:42.313ZDEBGtest_utilization (dropshot): registered endpoint local_addr = [::1]:39365 method = GET path = /support-bundles/{zpool_id}/{dataset_id}/{support_bundle_id}/download/{file} sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
8282025-04-30T06:43:42.313ZDEBGtest_utilization (dropshot): registered endpoint local_addr = [::1]:39365 method = HEAD path = /support-bundles/{zpool_id}/{dataset_id}/{support_bundle_id}/download/{file} sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
8292025-04-30T06:43:42.313ZDEBGtest_utilization (dropshot): registered endpoint local_addr = [::1]:39365 method = GET path = /support-bundles/{zpool_id}/{dataset_id}/{support_bundle_id}/index sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
8302025-04-30T06:43:42.313ZDEBGtest_utilization (dropshot): registered endpoint local_addr = [::1]:39365 method = HEAD path = /support-bundles/{zpool_id}/{dataset_id}/{support_bundle_id}/index sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
8312025-04-30T06:43:42.313ZDEBGtest_utilization (dropshot): registered endpoint local_addr = [::1]:39365 method = POST path = /switch-ports sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
8322025-04-30T06:43:42.313ZDEBGtest_utilization (dropshot): registered endpoint local_addr = [::1]:39365 method = GET path = /timesync sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
8332025-04-30T06:43:42.313ZDEBGtest_utilization (dropshot): registered endpoint local_addr = [::1]:39365 method = DELETE path = /v2p sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
8342025-04-30T06:43:42.313ZDEBGtest_utilization (dropshot): registered endpoint local_addr = [::1]:39365 method = GET path = /v2p sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
8352025-04-30T06:43:42.313ZDEBGtest_utilization (dropshot): registered endpoint local_addr = [::1]:39365 method = PUT path = /v2p sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
8362025-04-30T06:43:42.313ZDEBGtest_utilization (dropshot): registered endpoint local_addr = [::1]:39365 method = DELETE path = /vmms/{propolis_id} sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
8372025-04-30T06:43:42.313ZDEBGtest_utilization (dropshot): registered endpoint local_addr = [::1]:39365 method = PUT path = /vmms/{propolis_id} sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
8382025-04-30T06:43:42.313ZDEBGtest_utilization (dropshot): registered endpoint local_addr = [::1]:39365 method = POST path = /vmms/{propolis_id}/disks/{disk_id}/snapshot sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
8392025-04-30T06:43:42.313ZDEBGtest_utilization (dropshot): registered endpoint local_addr = [::1]:39365 method = DELETE path = /vmms/{propolis_id}/external-ip sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
8402025-04-30T06:43:42.313ZDEBGtest_utilization (dropshot): registered endpoint local_addr = [::1]:39365 method = PUT path = /vmms/{propolis_id}/external-ip sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
8412025-04-30T06:43:42.313ZDEBGtest_utilization (dropshot): registered endpoint local_addr = [::1]:39365 method = POST path = /vmms/{propolis_id}/poke sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
8422025-04-30T06:43:42.313ZDEBGtest_utilization (dropshot): registered endpoint local_addr = [::1]:39365 method = POST path = /vmms/{propolis_id}/poke-single-step sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
8432025-04-30T06:43:42.313ZDEBGtest_utilization (dropshot): registered endpoint local_addr = [::1]:39365 method = POST path = /vmms/{propolis_id}/sim-migration-source sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
8442025-04-30T06:43:42.313ZDEBGtest_utilization (dropshot): registered endpoint local_addr = [::1]:39365 method = GET path = /vmms/{propolis_id}/state sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
8452025-04-30T06:43:42.313ZDEBGtest_utilization (dropshot): registered endpoint local_addr = [::1]:39365 method = PUT path = /vmms/{propolis_id}/state sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
8462025-04-30T06:43:42.313ZDEBGtest_utilization (dropshot): registered endpoint local_addr = [::1]:39365 method = PUT path = /vpc/{vpc_id}/firewall/rules sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
8472025-04-30T06:43:42.313ZDEBGtest_utilization (dropshot): registered endpoint local_addr = [::1]:39365 method = GET path = /vpc-routes sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
8482025-04-30T06:43:42.313ZDEBGtest_utilization (dropshot): registered endpoint local_addr = [::1]:39365 method = PUT path = /vpc-routes sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
8492025-04-30T06:43:42.313ZDEBGtest_utilization (dropshot): registered endpoint local_addr = [::1]:39365 method = GET path = /zones sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
8502025-04-30T06:43:42.313ZDEBGtest_utilization (dropshot): registered endpoint local_addr = [::1]:39365 method = POST path = /zones/bundle-cleanup sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
8512025-04-30T06:43:42.313ZDEBGtest_utilization (dropshot): registered endpoint local_addr = [::1]:39365 method = GET path = /zones/bundle-cleanup/context sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
8522025-04-30T06:43:42.313ZDEBGtest_utilization (dropshot): registered endpoint local_addr = [::1]:39365 method = PUT path = /zones/bundle-cleanup/context sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
8532025-04-30T06:43:42.313ZDEBGtest_utilization (dropshot): registered endpoint local_addr = [::1]:39365 method = GET path = /zones/bundle-cleanup/utilization sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
8542025-04-30T06:43:42.313ZDEBGtest_utilization (dropshot): registered endpoint local_addr = [::1]:39365 method = GET path = /zones/bundles sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
8552025-04-30T06:43:42.313ZDEBGtest_utilization (dropshot): registered endpoint local_addr = [::1]:39365 method = GET path = /zones/bundles/{zone_name} sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
8562025-04-30T06:43:42.313ZDEBGtest_utilization (dropshot): registered endpoint local_addr = [::1]:39365 method = DELETE path = /zones/bundles/{zone_name}/{bundle_id} sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
8572025-04-30T06:43:42.313ZDEBGtest_utilization (dropshot): registered endpoint local_addr = [::1]:39365 method = GET path = /zones/bundles/{zone_name}/{bundle_id} sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
8582025-04-30T06:43:42.313ZDEBGtest_utilization (dropshot): registered endpoint local_addr = [::1]:39365 method = GET path = /zpools sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
8592025-04-30T06:43:42.313ZINFOtest_utilization (dropshot): listening local_addr = [::1]:39365 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
8602025-04-30T06:43:42.313ZDEBGtest_utilization (dropshot): successfully registered DTrace USDT probes local_addr = [::1]:39365 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
8612025-04-30T06:43:42.313ZDEBGtest_utilization (omicron_sled_agent::sim::Server): contacting server nexus sled_id = b6d65341-167c-41df-9b5c-41cded99c229
8622025-04-30T06:43:42.314ZDEBGtest_utilization (NexusClient): client request body = Some(Body) method = POST sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = http://127.0.0.1:32956/sled-agents/b6d65341-167c-41df-9b5c-41cded99c229
8632025-04-30T06:43:42.314ZTRACtest_utilization (dns): buffer buffer = Length: 75 (0x4b) bytes\n0000: b8 9e 01 00 00 01 00 00 00 00 00 01 0c 5f 6e 61 ............._na\n0010: 6d 65 73 65 72 76 69 63 65 04 5f 74 63 70 0d 63 meservice._tcp.c\n0020: 6f 6e 74 72 6f 6c 2d 70 6c 61 6e 65 05 6f 78 69 ontrol-plane.oxi\n0030: 64 65 08 69 6e 74 65 72 6e 61 6c 00 00 21 00 01 de.internal..!..\n0040: 00 00 29 04 d0 00 00 00 00 00 00 ..)........ kind = dns peer_addr = [::1]:56894 req_id = c998d70d-dbb7-4c17-a917-83c7f5c6c38c
8642025-04-30T06:43:42.315ZDEBGtest_utilization (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 47262,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_nameservice._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_nameservice._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 12,\n 95,\n 110,\n 97,\n 109,\n 101,\n 115,\n 101,\n 114,\n 118,\n 105,\n 99,\n 101,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:56894 req_id = c998d70d-dbb7-4c17-a917-83c7f5c6c38c
8652025-04-30T06:43:42.316ZERROtest_utilization (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 47262, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 1, }, query: WireQuery { query: LowerQuery { name: LowerName( Name("_nameservice._tcp.control-plane.oxide.internal."), ), original: Query { name: Name("_nameservice._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN, }, }, original: [ 12, 95, 110, 97, 109, 101, 115, 101, 114, 118, 105, 99, 101, 4, 95, 116, 99, 112, 13, 99, 111, 110, 116, 114, 111, 108, 45, 112, 108, 97, 110, 101, 5, 111, 120, 105, 100, 101, 8, 105, 110, 116, 101, 114, 110, 97, 108, 0, 0, 33, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: Some( Edns { rcode_high: 0, version: 0, dnssec_ok: false, max_payload: 1232, options: OPT { options: {}, }, }, ), } SERVFAIL: server is not authoritative for name: "_nameservice._tcp.control-plane.oxide.internal." kind = dns peer_addr = [::1]:56894 req_id = c998d70d-dbb7-4c17-a917-83c7f5c6c38c
8662025-04-30T06:43:42.316ZTRACtest_utilization (BackgroundTasks): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig
8672025-04-30T06:43:42.316ZTRACtest_utilization (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = dns_config_external dns_group = external resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
8682025-04-30T06:43:42.316ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database
8692025-04-30T06:43:42.316ZDEBGtest_utilization (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
8702025-04-30T06:43:42.317ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
8712025-04-30T06:43:42.318ZINFOtest_utilization (dropshot_internal): accepted connection local_addr = 127.0.0.1:32956 remote_addr = 127.0.0.1:64628
8722025-04-30T06:43:42.318ZWARNtest_utilization (BackgroundTasks): failed to lookup DNS servers background_task = dns_servers_internal dns_group = internal error = no record found for Query { name: Name("_nameservice._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }
8732025-04-30T06:43:42.318ZDEBGtest_utilization (BackgroundTasks): activation complete background_task = dns_servers_internal dns_group = internal elapsed = 95.031174ms iteration = 1 status = {"error":"failed to read list of DNS servers: no record found for Query { name: Name(\\"_nameservice._tcp.control-plane.oxide.internal.\\"), query_type: SRV, query_class: IN }"}
8742025-04-30T06:43:42.318ZTRACtest_utilization (dns): buffer buffer = Length: 76 (0x4c) bytes\n0000: 44 85 01 00 00 01 00 00 00 00 00 01 0d 5f 65 78 D............_ex\n0010: 74 65 72 6e 61 6c 2d 64 6e 73 04 5f 74 63 70 0d ternal-dns._tcp.\n0020: 63 6f 6e 74 72 6f 6c 2d 70 6c 61 6e 65 05 6f 78 control-plane.ox\n0030: 69 64 65 08 69 6e 74 65 72 6e 61 6c 00 00 21 00 ide.internal..!.\n0040: 01 00 00 29 04 d0 00 00 00 00 00 00 ...)........ kind = dns peer_addr = [::1]:54183 req_id = 8eeb02d5-2331-4bbf-b645-81c6a4ae1278
8752025-04-30T06:43:42.318ZDEBGtest_utilization (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 17541,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_external-dns._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_external-dns._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 13,\n 95,\n 101,\n 120,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 45,\n 100,\n 110,\n 115,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:54183 req_id = 8eeb02d5-2331-4bbf-b645-81c6a4ae1278
8762025-04-30T06:43:42.318ZERROtest_utilization (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 17541, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 1, }, query: WireQuery { query: LowerQuery { name: LowerName( Name("_external-dns._tcp.control-plane.oxide.internal."), ), original: Query { name: Name("_external-dns._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN, }, }, original: [ 13, 95, 101, 120, 116, 101, 114, 110, 97, 108, 45, 100, 110, 115, 4, 95, 116, 99, 112, 13, 99, 111, 110, 116, 114, 111, 108, 45, 112, 108, 97, 110, 101, 5, 111, 120, 105, 100, 101, 8, 105, 110, 116, 101, 114, 110, 97, 108, 0, 0, 33, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: Some( Edns { rcode_high: 0, version: 0, dnssec_ok: false, max_payload: 1232, options: OPT { options: {}, }, }, ), } SERVFAIL: server is not authoritative for name: "_external-dns._tcp.control-plane.oxide.internal." kind = dns peer_addr = [::1]:54183 req_id = 8eeb02d5-2331-4bbf-b645-81c6a4ae1278
8772025-04-30T06:43:42.319ZWARNtest_utilization (BackgroundTasks): failed to lookup DNS servers background_task = dns_servers_external dns_group = external error = no record found for Query { name: Name("_external-dns._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }
8782025-04-30T06:43:42.319ZDEBGtest_utilization (BackgroundTasks): activation complete background_task = dns_servers_external dns_group = external elapsed = 94.627499ms iteration = 1 status = {"error":"failed to read list of DNS servers: no record found for Query { name: Name(\\"_external-dns._tcp.control-plane.oxide.internal.\\"), query_type: SRV, query_class: IN }"}
8792025-04-30T06:43:42.322ZTRACtest_utilization (dropshot_internal): incoming request local_addr = 127.0.0.1:32956 method = POST remote_addr = 127.0.0.1:64628 req_id = ef8ccf44-c9e6-47f3-a2cf-24876ea80b7a uri = /sled-agents/b6d65341-167c-41df-9b5c-41cded99c229
8802025-04-30T06:43:42.325ZINFOtest_utilization (nexus): registered sled agent sled_uuid = b6d65341-167c-41df-9b5c-41cded99c229
8812025-04-30T06:43:42.332ZINFOtest_utilization (BackgroundTasks): Metric producer GC complete (no errors) background_task = metrics_producer_gc expiration = 2025-04-30 06:33:42.225433951 UTC pruned = {}
8822025-04-30T06:43:42.333ZDEBGtest_utilization (BackgroundTasks): activation complete background_task = metrics_producer_gc elapsed = 107.770446ms iteration = 1 status = {"expiration":"2025-04-30T06:33:42.225433951Z","pruned":[]}
8832025-04-30T06:43:42.334ZTRACtest_utilization (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal
8842025-04-30T06:43:42.340ZTRACtest_utilization (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal
8852025-04-30T06:43:42.341ZTRACtest_utilization (dns): buffer buffer = Length: 72 (0x48) bytes\n0000: f1 1a 01 00 00 01 00 00 00 00 00 01 09 5f 64 65 ............._de\n0010: 6e 64 72 69 74 65 04 5f 74 63 70 0d 63 6f 6e 74 ndrite._tcp.cont\n0020: 72 6f 6c 2d 70 6c 61 6e 65 05 6f 78 69 64 65 08 rol-plane.oxide.\n0030: 69 6e 74 65 72 6e 61 6c 00 00 21 00 01 00 00 29 internal..!....)\n0040: 04 d0 00 00 00 00 00 00 ........ kind = dns peer_addr = [::1]:60478 req_id = db5ff19e-1499-431b-9ac3-40667d44b261
8862025-04-30T06:43:42.341ZDEBGtest_utilization (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 61722,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_dendrite._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_dendrite._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 9,\n 95,\n 100,\n 101,\n 110,\n 100,\n 114,\n 105,\n 116,\n 101,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:60478 req_id = db5ff19e-1499-431b-9ac3-40667d44b261
8872025-04-30T06:43:42.341ZERROtest_utilization (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 61722, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 1, }, query: WireQuery { query: LowerQuery { name: LowerName( Name("_dendrite._tcp.control-plane.oxide.internal."), ), original: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN, }, }, original: [ 9, 95, 100, 101, 110, 100, 114, 105, 116, 101, 4, 95, 116, 99, 112, 13, 99, 111, 110, 116, 114, 111, 108, 45, 112, 108, 97, 110, 101, 5, 111, 120, 105, 100, 101, 8, 105, 110, 116, 101, 114, 110, 97, 108, 0, 0, 33, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: Some( Edns { rcode_high: 0, version: 0, dnssec_ok: false, max_payload: 1232, options: OPT { options: {}, }, }, ), } SERVFAIL: server is not authoritative for name: "_dendrite._tcp.control-plane.oxide.internal." kind = dns peer_addr = [::1]:60478 req_id = db5ff19e-1499-431b-9ac3-40667d44b261
8882025-04-30T06:43:42.341ZTRACtest_utilization (dns): buffer buffer = Length: 72 (0x48) bytes\n0000: 2b b0 01 00 00 01 00 00 00 00 00 01 09 5f 64 65 +............_de\n0010: 6e 64 72 69 74 65 04 5f 74 63 70 0d 63 6f 6e 74 ndrite._tcp.cont\n0020: 72 6f 6c 2d 70 6c 61 6e 65 05 6f 78 69 64 65 08 rol-plane.oxide.\n0030: 69 6e 74 65 72 6e 61 6c 00 00 21 00 01 00 00 29 internal..!....)\n0040: 04 d0 00 00 00 00 00 00 ........ kind = dns peer_addr = [::1]:60523 req_id = e29b719f-e571-4f7a-b42c-d67d06a2277b
8892025-04-30T06:43:42.342ZDEBGtest_utilization (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 11184,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_dendrite._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_dendrite._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 9,\n 95,\n 100,\n 101,\n 110,\n 100,\n 114,\n 105,\n 116,\n 101,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:60523 req_id = e29b719f-e571-4f7a-b42c-d67d06a2277b
8902025-04-30T06:43:42.343ZERROtest_utilization (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 11184, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 1, }, query: WireQuery { query: LowerQuery { name: LowerName( Name("_dendrite._tcp.control-plane.oxide.internal."), ), original: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN, }, }, original: [ 9, 95, 100, 101, 110, 100, 114, 105, 116, 101, 4, 95, 116, 99, 112, 13, 99, 111, 110, 116, 114, 111, 108, 45, 112, 108, 97, 110, 101, 5, 111, 120, 105, 100, 101, 8, 105, 110, 116, 101, 114, 110, 97, 108, 0, 0, 33, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: Some( Edns { rcode_high: 0, version: 0, dnssec_ok: false, max_payload: 1232, options: OPT { options: {}, }, }, ), } SERVFAIL: server is not authoritative for name: "_dendrite._tcp.control-plane.oxide.internal." kind = dns peer_addr = [::1]:60523 req_id = e29b719f-e571-4f7a-b42c-d67d06a2277b
8912025-04-30T06:43:42.343ZERROtest_utilization (BackgroundTasks): failed to resolve addresses for Dendrite services background_task = nat_v4_garbage_collector error = no record found for Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }
8922025-04-30T06:43:42.343ZERROtest_utilization (BackgroundTasks): failed to resolve addresses for Dendrite services background_task = nat_v4_garbage_collector error = no record found for Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }
8932025-04-30T06:43:42.343ZDEBGtest_utilization (BackgroundTasks): activation complete background_task = nat_v4_garbage_collector elapsed = 116.941268ms iteration = 1 status = {"error":"failed to resolve addresses for Dendrite services: no record found for Query { name: Name(\\"_dendrite._tcp.control-plane.oxide.internal.\\"), query_type: SRV, query_class: IN }"}
8942025-04-30T06:43:42.344ZERROtest_utilization (BackgroundTasks): failed to resolve addresses for Dendrite services background_task = bfd_manager error = no record found for Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }
8952025-04-30T06:43:42.344ZERROtest_utilization (BackgroundTasks): failed to resolve addresses for Dendrite services background_task = bfd_manager error = no record found for Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }
8962025-04-30T06:43:42.344ZDEBGtest_utilization (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 113.118261ms iteration = 1 status = {"error":"failed to resolve addresses for Dendrite services: no record found for Query { name: Name(\\"_dendrite._tcp.control-plane.oxide.internal.\\"), query_type: SRV, query_class: IN }"}
8972025-04-30T06:43:42.361ZDEBGtest_utilization (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
8982025-04-30T06:43:42.369ZDEBGtest_utilization (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
8992025-04-30T06:43:42.369ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database
9002025-04-30T06:43:42.369ZDEBGtest_utilization (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
9012025-04-30T06:43:42.369ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
9022025-04-30T06:43:42.370ZDEBGtest_utilization (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 137.868414ms iteration = 1 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
9032025-04-30T06:43:42.415ZDEBGtest_utilization (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
9042025-04-30T06:43:42.421ZDEBGtest_utilization (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
9052025-04-30T06:43:42.421ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database
9062025-04-30T06:43:42.421ZDEBGtest_utilization (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
9072025-04-30T06:43:42.422ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
9082025-04-30T06:43:42.422ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
9092025-04-30T06:43:42.427ZDEBGtest_utilization (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory result = Ok(())
9102025-04-30T06:43:42.427ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
9112025-04-30T06:43:42.427ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
9122025-04-30T06:43:42.427ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
9132025-04-30T06:43:42.490ZDEBGtest_utilization (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
9142025-04-30T06:43:42.494ZDEBGtest_utilization (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
9152025-04-30T06:43:42.494ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database
9162025-04-30T06:43:42.494ZDEBGtest_utilization (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
9172025-04-30T06:43:42.494ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
9182025-04-30T06:43:42.516ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database
9192025-04-30T06:43:42.516ZDEBGtest_utilization (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
9202025-04-30T06:43:42.517ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
9212025-04-30T06:43:42.519ZDEBGtest_utilization (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
9222025-04-30T06:43:42.522ZDEBGtest_utilization (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
9232025-04-30T06:43:42.522ZTRACtest_utilization (BackgroundTasks): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet
9242025-04-30T06:43:42.522ZTRACtest_utilization (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = vpc_route_manager resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
9252025-04-30T06:43:42.522ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database
9262025-04-30T06:43:42.522ZDEBGtest_utilization (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
9272025-04-30T06:43:42.523ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
9282025-04-30T06:43:42.650ZDEBGtest_utilization (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
9292025-04-30T06:43:42.654ZDEBGtest_utilization (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
9302025-04-30T06:43:42.654ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database
9312025-04-30T06:43:42.654ZDEBGtest_utilization (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
9322025-04-30T06:43:42.654ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
9332025-04-30T06:43:42.676ZDEBGtest_utilization (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
9342025-04-30T06:43:42.679ZDEBGtest_utilization (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Fleet result = Ok(())
9352025-04-30T06:43:42.679ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database
9362025-04-30T06:43:42.679ZDEBGtest_utilization (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
9372025-04-30T06:43:42.680ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
9382025-04-30T06:43:42.680ZDEBGtest_utilization (BackgroundTasks): activation complete background_task = lookup_region_port elapsed = 435.540526ms iteration = 1 status = {"errors":[],"found_port_ok":[]}
9392025-04-30T06:43:42.683ZDEBGtest_utilization (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
9402025-04-30T06:43:42.687ZDEBGtest_utilization (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
9412025-04-30T06:43:42.687ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database
9422025-04-30T06:43:42.687ZDEBGtest_utilization (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
9432025-04-30T06:43:42.687ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
9442025-04-30T06:43:42.728ZDEBGtest_utilization (BackgroundTasks): roles background_task = tuf_artifact_replication roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
9452025-04-30T06:43:42.733ZDEBGtest_utilization (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = tuf_artifact_replication resource = Fleet result = Ok(())
9462025-04-30T06:43:42.733ZTRACtest_utilization (BackgroundTasks): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = tuf_artifact_replication resource = Fleet
9472025-04-30T06:43:42.733ZTRACtest_utilization (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = tuf_artifact_replication resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
9482025-04-30T06:43:42.733ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = tuf_artifact_replication resource = Database
9492025-04-30T06:43:42.733ZDEBGtest_utilization (BackgroundTasks): roles background_task = tuf_artifact_replication roles = RoleSet { roles: {} }
9502025-04-30T06:43:42.734ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = tuf_artifact_replication resource = Database result = Ok(())
9512025-04-30T06:43:42.791ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database
9522025-04-30T06:43:42.791ZDEBGtest_utilization (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
9532025-04-30T06:43:42.792ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
9542025-04-30T06:43:42.830ZDEBGtest_utilization (BackgroundTasks): activation complete background_task = webhook_deliverator elapsed = 578.852776ms iteration = 1 status = {"by_rx":{},"error":null}
9552025-04-30T06:43:42.832ZDEBGtest_utilization (BackgroundTasks): Found abandoned VMMs background_task = abandoned_vmm_reaper count = 0 total = 0
9562025-04-30T06:43:42.832ZINFOtest_utilization (BackgroundTasks): Abandoned VMMs reaped background_task = abandoned_vmm_reaper sled_reservations_deleted = 0 vmms_already_deleted = 0 vmms_deleted = 0 vmms_found = 0
9572025-04-30T06:43:42.832ZDEBGtest_utilization (BackgroundTasks): activation complete background_task = abandoned_vmm_reaper elapsed = 589.270263ms iteration = 1 status = {"errors":[],"sled_reservations_deleted":0,"vmms_already_deleted":0,"vmms_deleted":0,"vmms_found":0}
9582025-04-30T06:43:42.884ZDEBGtest_utilization (BackgroundTasks): activation complete background_task = region_snapshot_replacement_garbage_collection elapsed = 638.351013ms iteration = 1 status = {"errors":[],"garbage_collect_requested":[]}
9592025-04-30T06:43:42.894ZDEBGtest_utilization (BackgroundTasks): activation complete background_task = region_snapshot_replacement_finish elapsed = 644.722047ms iteration = 1 status = {"errors":[],"finish_invoked_ok":[]}
9602025-04-30T06:43:42.906ZINFOtest_utilization (SagaRecoverer): listed in-progress sagas count = 0
9612025-04-30T06:43:42.906ZDEBGtest_utilization (BackgroundTasks): activation complete background_task = saga_recovery elapsed = 662.353026ms iteration = 1 status = {"last_pass":{"Success":{"nfailed":0,"nfound":0,"nrecovered":0,"nremoved":0,"nskipped":0}},"ntotal_failures":0,"ntotal_finished":0,"ntotal_recovered":0,"ntotal_sec_errors_bad_state":0,"ntotal_sec_errors_missing":0,"ntotal_started":0,"recent_failures":[],"recent_recoveries":[]}
9622025-04-30T06:43:42.906ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database
9632025-04-30T06:43:42.906ZDEBGtest_utilization (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
9642025-04-30T06:43:42.907ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
9652025-04-30T06:43:42.907ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database
9662025-04-30T06:43:42.907ZDEBGtest_utilization (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
9672025-04-30T06:43:42.909ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
9682025-04-30T06:43:42.928ZDEBGtest_utilization (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
9692025-04-30T06:43:42.937ZDEBGtest_utilization (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
9702025-04-30T06:43:42.972ZTRACtest_utilization (BackgroundTasks): no more instances in need of reincarnation background_task = instance_reincarnation reincarnation_reason = instance failed total_found = 0
9712025-04-30T06:43:42.972ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_reincarnation resource = Database
9722025-04-30T06:43:42.972ZDEBGtest_utilization (BackgroundTasks): roles background_task = instance_reincarnation roles = RoleSet { roles: {} }
9732025-04-30T06:43:42.973ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_reincarnation resource = Database result = Ok(())
9742025-04-30T06:43:42.994ZDEBGtest_utilization (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
9752025-04-30T06:43:43.000ZDEBGtest_utilization (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
9762025-04-30T06:43:43.047ZDEBGtest_utilization (BackgroundTasks): activation complete background_task = read_only_region_replacement_start elapsed = 797.266932ms iteration = 1 status = {"errors":[],"requests_created_ok":[]}
9772025-04-30T06:43:43.054ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database
9782025-04-30T06:43:43.054ZDEBGtest_utilization (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
9792025-04-30T06:43:43.055ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
9802025-04-30T06:43:43.125ZWARNtest_utilization (BackgroundTasks): failed to read DNS config background_task = dns_config_external dns_group = external error = Internal Error: expected exactly one latest version for DNS group External, found 0
9812025-04-30T06:43:43.125ZDEBGtest_utilization (BackgroundTasks): activation complete background_task = dns_config_external dns_group = external elapsed = 901.524208ms iteration = 1 status = {"error":"failed to read DNS config: Internal Error: expected exactly one latest version for DNS group External, found 0"}
9822025-04-30T06:43:43.126ZWARNtest_utilization (BackgroundTasks): failed to read target blueprint background_task = blueprint_loader error = Internal Error: no target blueprint set
9832025-04-30T06:43:43.126ZDEBGtest_utilization (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 894.056409ms iteration = 1 status = {"error":"failed to read target blueprint: Internal Error: no target blueprint set"}
9842025-04-30T06:43:43.258ZTRACtest_utilization (BackgroundTasks): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Fleet
9852025-04-30T06:43:43.258ZTRACtest_utilization (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = support_bundle_collector resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
9862025-04-30T06:43:43.258ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database
9872025-04-30T06:43:43.258ZDEBGtest_utilization (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
9882025-04-30T06:43:43.259ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
9892025-04-30T06:43:43.269ZTRACtest_utilization (BackgroundTasks): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = DnsConfig
9902025-04-30T06:43:43.269ZTRACtest_utilization (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = external_endpoints resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
9912025-04-30T06:43:43.269ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database
9922025-04-30T06:43:43.269ZDEBGtest_utilization (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
9932025-04-30T06:43:43.270ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
9942025-04-30T06:43:43.270ZDEBGtest_utilization (BackgroundTasks): inventory_prune_one: nothing eligible for removal (too few) background_task = inventory_collection candidates = []
9952025-04-30T06:43:43.270ZTRACtest_utilization (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal
9962025-04-30T06:43:43.270ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database
9972025-04-30T06:43:43.270ZDEBGtest_utilization (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
9982025-04-30T06:43:43.271ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
9992025-04-30T06:43:43.277ZDEBGtest_utilization (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
10002025-04-30T06:43:43.282ZDEBGtest_utilization (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
10012025-04-30T06:43:43.282ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database
10022025-04-30T06:43:43.283ZDEBGtest_utilization (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
10032025-04-30T06:43:43.283ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
10042025-04-30T06:43:43.283ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database
10052025-04-30T06:43:43.283ZDEBGtest_utilization (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
10062025-04-30T06:43:43.284ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
10072025-04-30T06:43:43.284ZTRACtest_utilization (dns): buffer buffer = Length: 67 (0x43) bytes\n0000: c6 81 01 00 00 01 00 00 00 00 00 01 04 5f 6d 67 ............._mg\n0010: 73 04 5f 74 63 70 0d 63 6f 6e 74 72 6f 6c 2d 70 s._tcp.control-p\n0020: 6c 61 6e 65 05 6f 78 69 64 65 08 69 6e 74 65 72 lane.oxide.inter\n0030: 6e 61 6c 00 00 21 00 01 00 00 29 04 d0 00 00 00 nal..!....).....\n0040: 00 00 00 ... kind = dns peer_addr = [::1]:56459 req_id = 3458e7b3-c6b3-4dfa-934f-c5fc7ee0f522
10082025-04-30T06:43:43.284ZDEBGtest_utilization (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 50817,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_mgs._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_mgs._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 4,\n 95,\n 109,\n 103,\n 115,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:56459 req_id = 3458e7b3-c6b3-4dfa-934f-c5fc7ee0f522
10092025-04-30T06:43:43.285ZERROtest_utilization (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 50817, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 1, }, query: WireQuery { query: LowerQuery { name: LowerName( Name("_mgs._tcp.control-plane.oxide.internal."), ), original: Query { name: Name("_mgs._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN, }, }, original: [ 4, 95, 109, 103, 115, 4, 95, 116, 99, 112, 13, 99, 111, 110, 116, 114, 111, 108, 45, 112, 108, 97, 110, 101, 5, 111, 120, 105, 100, 101, 8, 105, 110, 116, 101, 114, 110, 97, 108, 0, 0, 33, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: Some( Edns { rcode_high: 0, version: 0, dnssec_ok: false, max_payload: 1232, options: OPT { options: {}, }, }, ), } SERVFAIL: server is not authoritative for name: "_mgs._tcp.control-plane.oxide.internal." kind = dns peer_addr = [::1]:56459 req_id = 3458e7b3-c6b3-4dfa-934f-c5fc7ee0f522
10102025-04-30T06:43:43.285ZWARNtest_utilization (BackgroundTasks): inventory collection failed background_task = inventory_collection error = failed to collect inventory: looking up MGS addresses: no record found for Query { name: Name("_mgs._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }
10112025-04-30T06:43:43.285ZDEBGtest_utilization (BackgroundTasks): activation complete background_task = inventory_collection elapsed = 1.052843933s iteration = 1 status = {"error":"failed to collect inventory: looking up MGS addresses: no record found for Query { name: Name(\\"_mgs._tcp.control-plane.oxide.internal.\\"), query_type: SRV, query_class: IN }"}
10122025-04-30T06:43:43.302ZDEBGtest_utilization (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 1.067342611s iteration = 1 status = {}
10132025-04-30T06:43:43.302ZWARNtest_utilization (BackgroundTasks): failed to read DNS config background_task = dns_config_internal dns_group = internal error = Internal Error: expected exactly one latest version for DNS group Internal, found 0
10142025-04-30T06:43:43.302ZDEBGtest_utilization (BackgroundTasks): activation complete background_task = dns_config_internal dns_group = internal elapsed = 1.081005333s iteration = 1 status = {"error":"failed to read DNS config: Internal Error: expected exactly one latest version for DNS group Internal, found 0"}
10152025-04-30T06:43:43.303ZDEBGtest_utilization (BackgroundTasks): activating background_task = inventory_collection iteration = 2 reason = Signaled
10162025-04-30T06:43:43.303ZTRACtest_utilization (BackgroundTasks): authorize begin action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory
10172025-04-30T06:43:43.303ZTRACtest_utilization (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
10182025-04-30T06:43:43.303ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
10192025-04-30T06:43:43.303ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
10202025-04-30T06:43:43.304ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
10212025-04-30T06:43:43.304ZINFOtest_utilization (dropshot_internal): request completed latency_us = 981635 local_addr = 127.0.0.1:32956 method = POST remote_addr = 127.0.0.1:64628 req_id = ef8ccf44-c9e6-47f3-a2cf-24876ea80b7a response_code = 204 uri = /sled-agents/b6d65341-167c-41df-9b5c-41cded99c229
10222025-04-30T06:43:43.304ZDEBGtest_utilization (NexusClient): client response result = Ok(Response { url: "http://127.0.0.1:32956/sled-agents/b6d65341-167c-41df-9b5c-41cded99c229", status: 204, headers: {"x-request-id": "ef8ccf44-c9e6-47f3-a2cf-24876ea80b7a", "date": "Wed, 30 Apr 2025 06:43:42 GMT"} }) sled_id = b6d65341-167c-41df-9b5c-41cded99c229
10232025-04-30T06:43:43.304ZDEBGtest_utilization (init_with_steps): Running step start_crucible_pantry
10242025-04-30T06:43:43.306ZDEBGtest_utilization (dropshot): registered endpoint kind = pantry local_addr = [::1]:45558 method = GET path = /crucible/pantry/0 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
10252025-04-30T06:43:43.306ZDEBGtest_utilization (dropshot): registered endpoint kind = pantry local_addr = [::1]:45558 method = GET path = /crucible/pantry/0/job/{id}/is-finished sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
10262025-04-30T06:43:43.306ZDEBGtest_utilization (dropshot): registered endpoint kind = pantry local_addr = [::1]:45558 method = GET path = /crucible/pantry/0/job/{id}/ok sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
10272025-04-30T06:43:43.306ZDEBGtest_utilization (dropshot): registered endpoint kind = pantry local_addr = [::1]:45558 method = DELETE path = /crucible/pantry/0/volume/{id} sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
10282025-04-30T06:43:43.306ZDEBGtest_utilization (dropshot): registered endpoint kind = pantry local_addr = [::1]:45558 method = GET path = /crucible/pantry/0/volume/{id} sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
10292025-04-30T06:43:43.306ZDEBGtest_utilization (dropshot): registered endpoint kind = pantry local_addr = [::1]:45558 method = POST path = /crucible/pantry/0/volume/{id} sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
10302025-04-30T06:43:43.306ZDEBGtest_utilization (dropshot): registered endpoint kind = pantry local_addr = [::1]:45558 method = POST path = /crucible/pantry/0/volume/{id}/background sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
10312025-04-30T06:43:43.306ZDEBGtest_utilization (dropshot): registered endpoint kind = pantry local_addr = [::1]:45558 method = POST path = /crucible/pantry/0/volume/{id}/bulk-write sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
10322025-04-30T06:43:43.306ZDEBGtest_utilization (dropshot): registered endpoint kind = pantry local_addr = [::1]:45558 method = POST path = /crucible/pantry/0/volume/{id}/import-from-url sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
10332025-04-30T06:43:43.306ZDEBGtest_utilization (dropshot): registered endpoint kind = pantry local_addr = [::1]:45558 method = POST path = /crucible/pantry/0/volume/{id}/scrub sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
10342025-04-30T06:43:43.306ZDEBGtest_utilization (dropshot): registered endpoint kind = pantry local_addr = [::1]:45558 method = POST path = /crucible/pantry/0/volume/{id}/snapshot sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
10352025-04-30T06:43:43.306ZINFOtest_utilization (dropshot): listening kind = pantry local_addr = [::1]:45558 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
10362025-04-30T06:43:43.307ZDEBGtest_utilization (dropshot): successfully registered DTrace USDT probes kind = pantry local_addr = [::1]:45558 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
10372025-04-30T06:43:43.307ZINFOtest_utilization (omicron_sled_agent::sim::Server): Started Simulated Crucible Pantry address = [::1]:45558 kind = pantry sled_id = b6d65341-167c-41df-9b5c-41cded99c229
10382025-04-30T06:43:43.307ZDEBGtest_utilization (init_with_steps): Running step populate_internal_dns
10392025-04-30T06:43:43.307ZDEBGtest_utilization: Populating Internal DNS
10402025-04-30T06:43:43.361ZINFOtest_utilization: DNS population: DnsConfigParams { generation: Generation( 1, ), time_created: 2025-04-30T06:43:43.361315952Z, zones: [ DnsConfigZone { zone_name: "control-plane.oxide.internal", records: { "_clickhouse-admin-single-server._tcp": [ Srv( Srv { prio: 0, weight: 0, port: 8888, target: "f2ab2f6d-2924-4302-a5e6-5e7be89815fe.host.control-plane.oxide.internal", }, ), ], "dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host": [ Aaaa( ::1, ), ], "613f608b-b7cb-4bec-8ce6-ee487de665d8.host": [ Aaaa( ::1, ), ], "2ec6b9d0-21c7-4ff3-a311-7a18b348aa25.host": [ Aaaa( ::1, ), ], "aa018865-577e-46e1-88d5-a00bfb387813.host": [ Aaaa( ::1, ), ], "f2ab2f6d-2924-4302-a5e6-5e7be89815fe.host": [ Aaaa( ::1, ), ], "_clickhouse-native._tcp": [ Srv( Srv { prio: 0, weight: 0, port: 9000, target: "f2ab2f6d-2924-4302-a5e6-5e7be89815fe.host.control-plane.oxide.internal", }, ), ], "_clickhouse._tcp": [ Srv( Srv { prio: 0, weight: 0, port: 44605, target: "f2ab2f6d-2924-4302-a5e6-5e7be89815fe.host.control-plane.oxide.internal", }, ), ], "b6d65341-167c-41df-9b5c-41cded99c229.sled": [ Aaaa( ::1, ), ], "_cockroach._tcp": [ Srv( Srv { prio: 0, weight: 0, port: 65362, target: "aa018865-577e-46e1-88d5-a00bfb387813.host.control-plane.oxide.internal", }, ), ], "47c428f0-ed10-4268-b847-550bb644ba22.host": [ Aaaa( ::1, ), ], "_nexus._tcp": [ Srv( Srv { prio: 0, weight: 0, port: 32956, target: "ac55d8d9-a298-4534-898b-beee1c9e4e52.host.control-plane.oxide.internal", }, ), ], "_oximeter-reader._tcp": [ Srv( Srv { prio: 0, weight: 0, port: 9000, target: "f2ab2f6d-2924-4302-a5e6-5e7be89815fe.host.control-plane.oxide.internal", }, ), ], "_mgs._tcp": [ Srv( Srv { prio: 0, weight: 0, port: 48759, target: "dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal", }, ), ], "ac55d8d9-a298-4534-898b-beee1c9e4e52.host": [ Aaaa( ::ffff:127.0.0.1, ), ], "_repo-depot._tcp": [ Srv( Srv { prio: 0, weight: 0, port: 51171, target: "b6d65341-167c-41df-9b5c-41cded99c229.sled.control-plane.oxide.internal", }, ), ], "_nameservice._tcp": [ Srv( Srv { prio: 0, weight: 0, port: 42391, target: "2ec6b9d0-21c7-4ff3-a311-7a18b348aa25.host.control-plane.oxide.internal", }, ), ], "_external-dns._tcp": [ Srv( Srv { prio: 0, weight: 0, port: 48132, target: "47c428f0-ed10-4268-b847-550bb644ba22.host.control-plane.oxide.internal", }, ), ], "_dendrite._tcp": [ Srv( Srv { prio: 0, weight: 0, port: 47974, target: "dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal", }, ), ], "_crucible-pantry._tcp": [ Srv( Srv { prio: 0, weight: 0, port: 45558, target: "613f608b-b7cb-4bec-8ce6-ee487de665d8.host.control-plane.oxide.internal", }, ), ], "_mgd._tcp": [ Srv( Srv { prio: 0, weight: 0, port: 52102, target: "dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal", }, ), ], }, }, ], }
10412025-04-30T06:43:43.361ZDEBGtest_utilization: client request body = Some(Body) method = PUT uri = http://[::1]:42391/config
10422025-04-30T06:43:43.379ZINFOtest_utilization (http): accepted connection kind = dns local_addr = [::1]:42391 remote_addr = [::1]:54365
10432025-04-30T06:43:43.380ZTRACtest_utilization (http): incoming request kind = dns local_addr = [::1]:42391 method = PUT remote_addr = [::1]:54365 req_id = e3f83685-4b0f-4489-9164-83577ae1eb3a uri = /config
10442025-04-30T06:43:43.381ZINFOtest_utilization (store): attempting generation update new_generation = 1 req_id = e3f83685-4b0f-4489-9164-83577ae1eb3a
10452025-04-30T06:43:43.381ZINFOtest_utilization (store): pruning trees for generations newer than 0
10462025-04-30T06:43:43.381ZDEBGtest_utilization (store): creating tree tree_name = generation_1_zone_control-plane.oxide.internal
10472025-04-30T06:43:43.382ZDEBGtest_utilization (store): updating current config
10482025-04-30T06:43:43.382ZDEBGtest_utilization (store): flushing default tree
10492025-04-30T06:43:43.382ZINFOtest_utilization (store): pruning trees for generations older than 1 keep = 3
10502025-04-30T06:43:43.382ZINFOtest_utilization (store): updated generation new_generation = 1 req_id = e3f83685-4b0f-4489-9164-83577ae1eb3a
10512025-04-30T06:43:43.383ZINFOtest_utilization (http): request completed kind = dns latency_us = 2306 local_addr = [::1]:42391 method = PUT remote_addr = [::1]:54365 req_id = e3f83685-4b0f-4489-9164-83577ae1eb3a response_code = 204 uri = /config
10522025-04-30T06:43:43.383ZDEBGtest_utilization: client response result = Ok(Response { url: "http://[::1]:42391/config", status: 204, headers: {"x-request-id": "e3f83685-4b0f-4489-9164-83577ae1eb3a", "date": "Wed, 30 Apr 2025 06:43:43 GMT"} })
10532025-04-30T06:43:43.386ZDEBGtest_utilization (init_with_steps): Running step configure_sled_agents
10542025-04-30T06:43:43.435ZDEBGtest_utilization: client request body = Some(Body) method = PUT uri = http://[::1]:39365/omicron-config
10552025-04-30T06:43:43.442ZWARNtest_utilization (BackgroundTasks): inventory collection is None background_task = service_zone_nat_tracker
10562025-04-30T06:43:43.442ZDEBGtest_utilization (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 1.207957467s iteration = 1 status = {"error":"inventory collection is None"}
10572025-04-30T06:43:43.443ZINFOtest_utilization (dropshot): accepted connection local_addr = [::1]:39365 remote_addr = [::1]:56282 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
10582025-04-30T06:43:43.443ZTRACtest_utilization (dropshot): incoming request local_addr = [::1]:39365 method = PUT remote_addr = [::1]:56282 req_id = dc124c2d-f881-4148-99db-b7e67b861a6b sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /omicron-config
10592025-04-30T06:43:43.446ZINFOtest_utilization (dropshot): request completed latency_us = 2930 local_addr = [::1]:39365 method = PUT remote_addr = [::1]:56282 req_id = dc124c2d-f881-4148-99db-b7e67b861a6b response_code = 200 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /omicron-config
10602025-04-30T06:43:43.447ZDEBGtest_utilization (BackgroundTasks): roles background_task = tuf_artifact_replication roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
10612025-04-30T06:43:43.456ZDEBGtest_utilization (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = tuf_artifact_replication resource = Fleet result = Ok(())
10622025-04-30T06:43:43.456ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = tuf_artifact_replication resource = Database
10632025-04-30T06:43:43.456ZDEBGtest_utilization (BackgroundTasks): roles background_task = tuf_artifact_replication roles = RoleSet { roles: {} }
10642025-04-30T06:43:43.457ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = tuf_artifact_replication resource = Database result = Ok(())
10652025-04-30T06:43:43.460ZDEBGtest_utilization: client response result = Ok(Response { url: "http://[::1]:39365/omicron-config", status: 200, headers: {"content-type": "application/json", "x-request-id": "dc124c2d-f881-4148-99db-b7e67b861a6b", "content-length": "26", "date": "Wed, 30 Apr 2025 06:43:43 GMT"} })
10662025-04-30T06:43:43.461ZDEBGtest_utilization: client request body = Some(Body) method = PUT uri = http://[::1]:39365/network-bootstore-config
10672025-04-30T06:43:43.461ZTRACtest_utilization (dropshot): incoming request local_addr = [::1]:39365 method = PUT remote_addr = [::1]:56282 req_id = 802a7bbb-2b73-4d5f-9e3a-4ac9a549197b sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /network-bootstore-config
10682025-04-30T06:43:43.462ZINFOtest_utilization (dropshot): request completed latency_us = 836 local_addr = [::1]:39365 method = PUT remote_addr = [::1]:56282 req_id = 802a7bbb-2b73-4d5f-9e3a-4ac9a549197b response_code = 204 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /network-bootstore-config
10692025-04-30T06:43:43.463ZDEBGtest_utilization: client response result = Ok(Response { url: "http://[::1]:39365/network-bootstore-config", status: 204, headers: {"x-request-id": "802a7bbb-2b73-4d5f-9e3a-4ac9a549197b", "date": "Wed, 30 Apr 2025 06:43:43 GMT"} })
10702025-04-30T06:43:43.464ZDEBGtest_utilization (init_with_steps): Running step start_nexus_external
10712025-04-30T06:43:43.464ZDEBGtest_utilization: Starting Nexus (external API)
10722025-04-30T06:43:46.471ZTRACtest_utilization (InternalApi): authorize begin action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Fleet
10732025-04-30T06:43:46.471ZTRACtest_utilization (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
10742025-04-30T06:43:46.471ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
10752025-04-30T06:43:46.471ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
10762025-04-30T06:43:46.472ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
10772025-04-30T06:43:46.473ZTRACtest_utilization (BackgroundTasks): reprocessing 0 outdated webhook globs... background_task = webhook_dispatcher
10782025-04-30T06:43:46.473ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = webhook_dispatcher resource = Database
10792025-04-30T06:43:46.473ZDEBGtest_utilization (BackgroundTasks): roles background_task = webhook_dispatcher roles = RoleSet { roles: {} }
10802025-04-30T06:43:46.474ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = webhook_dispatcher resource = Database result = Ok(())
10812025-04-30T06:43:46.474ZINFOtest_utilization (BackgroundTasks): all instance checks complete background_task = instance_watcher pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0 update_sagas_queued = 0
10822025-04-30T06:43:46.475ZDEBGtest_utilization (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 4.236711853s iteration = 1 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{},"pruned_instances":0,"total_instances":0,"update_sagas_queued":0}
10832025-04-30T06:43:46.475ZDEBGtest_utilization (BackgroundTasks): activation complete background_task = region_replacement elapsed = 4.237990389s iteration = 1 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
10842025-04-30T06:43:46.475ZDEBGtest_utilization (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 4.237496556s iteration = 1 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
10852025-04-30T06:43:46.477ZDEBGtest_utilization (BackgroundTasks): activation complete background_task = region_snapshot_replacement_start elapsed = 4.232230714s iteration = 1 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
10862025-04-30T06:43:46.492ZDEBGtest_utilization (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
10872025-04-30T06:43:46.496ZDEBGtest_utilization (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
10882025-04-30T06:43:46.496ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database
10892025-04-30T06:43:46.496ZDEBGtest_utilization (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
10902025-04-30T06:43:46.497ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
10912025-04-30T06:43:46.497ZDEBGtest_utilization (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
10922025-04-30T06:43:46.505ZDEBGtest_utilization (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = DnsConfig result = Ok(())
10932025-04-30T06:43:46.505ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database
10942025-04-30T06:43:46.505ZDEBGtest_utilization (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
10952025-04-30T06:43:46.506ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
10962025-04-30T06:43:46.506ZTRACtest_utilization (BackgroundTasks): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) }
10972025-04-30T06:43:46.506ZTRACtest_utilization (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = service_firewall_rule_propagation resource_id = 001de000-4401-4000-8000-000000000000 resource_type = Project
10982025-04-30T06:43:46.506ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database
10992025-04-30T06:43:46.507ZDEBGtest_utilization (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
11002025-04-30T06:43:46.507ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
11012025-04-30T06:43:46.612ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
11022025-04-30T06:43:46.615ZDEBGtest_utilization (InternalApi): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Fleet result = Ok(())
11032025-04-30T06:43:46.615ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
11042025-04-30T06:43:46.615ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
11052025-04-30T06:43:46.616ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
11062025-04-30T06:43:46.639ZTRACtest_utilization (BackgroundTasks): webhook dispatching completed successfully background_task = webhook_dispatcher events_dispatched = 0 events_without_receivers = 0
11072025-04-30T06:43:46.640ZDEBGtest_utilization (BackgroundTasks): activation complete background_task = webhook_dispatcher elapsed = 4.389703489s iteration = 1 status = {"dispatched":[],"errors":[],"glob_version":"140.0.0","globs_reprocessed":{},"no_receivers":[]}
11082025-04-30T06:43:46.645ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database
11092025-04-30T06:43:46.645ZDEBGtest_utilization (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
11102025-04-30T06:43:46.645ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
11112025-04-30T06:43:46.691ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
11122025-04-30T06:43:46.714ZDEBGtest_utilization (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory result = Ok(())
11132025-04-30T06:43:46.714ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
11142025-04-30T06:43:46.714ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
11152025-04-30T06:43:46.715ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
11162025-04-30T06:43:46.743ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
11172025-04-30T06:43:46.777ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
11182025-04-30T06:43:46.779ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
11192025-04-30T06:43:46.780ZINFOtest_utilization (BackgroundTasks): No bundles to collect background_task = support_bundle_collector
11202025-04-30T06:43:46.780ZDEBGtest_utilization (BackgroundTasks): activation complete background_task = support_bundle_collector elapsed = 4.547166944s iteration = 1 status = {"cleanup_err":null,"cleanup_report":{"db_destroying_bundles_removed":0,"db_failing_bundles_updated":0,"sled_bundles_delete_failed":0,"sled_bundles_deleted_not_found":0,"sled_bundles_deleted_ok":0},"collection_err":null,"collection_report":null}
11212025-04-30T06:43:46.781ZWARNtest_utilization (BackgroundTasks): failed to read Silo/DNS/TLS configuration background_task = external_endpoints error = Internal Error: expected at least one external DNS zone
11222025-04-30T06:43:46.781ZDEBGtest_utilization (BackgroundTasks): activation complete background_task = external_endpoints elapsed = 4.554838148s iteration = 1 status = {"error":"failed to read Silo/DNS/TLS configuration: Internal Error: expected at least one external DNS zone"}
11232025-04-30T06:43:46.781ZTRACtest_utilization (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = service_firewall_rule_propagation resource_id = 001de000-5110-4000-8000-000000000001 resource_type = Silo
11242025-04-30T06:43:46.781ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database
11252025-04-30T06:43:46.781ZDEBGtest_utilization (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
11262025-04-30T06:43:46.781ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
11272025-04-30T06:43:46.875ZTRACtest_utilization (BackgroundTasks): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = tuf_artifact_replication resource = Fleet
11282025-04-30T06:43:46.875ZTRACtest_utilization (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = tuf_artifact_replication resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
11292025-04-30T06:43:46.875ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = tuf_artifact_replication resource = Database
11302025-04-30T06:43:46.875ZDEBGtest_utilization (BackgroundTasks): roles background_task = tuf_artifact_replication roles = RoleSet { roles: {} }
11312025-04-30T06:43:46.876ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = tuf_artifact_replication resource = Database result = Ok(())
11322025-04-30T06:43:46.978ZINFOtest_utilization (BackgroundTasks): VPC route manager sled b6d65341-167c-41df-9b5c-41cded99c229 background_task = vpc_route_manager
11332025-04-30T06:43:46.979ZDEBGtest_utilization (BackgroundTasks): client request SledAgent = b6d65341-167c-41df-9b5c-41cded99c229 background_task = vpc_route_manager body = None method = GET uri = http://[::1]:39365/vpc-routes
11342025-04-30T06:43:46.980ZTRACtest_utilization (BackgroundTasks): no more instances in need of reincarnation background_task = instance_reincarnation reincarnation_reason = start saga failed total_found = 0
11352025-04-30T06:43:46.980ZINFOtest_utilization (BackgroundTasks): instance reincarnation completed successfully background_task = instance_reincarnation instances_changed_state = 0 instances_found = 0 instances_reincarnated = 0
11362025-04-30T06:43:46.980ZDEBGtest_utilization (BackgroundTasks): activation complete background_task = instance_reincarnation elapsed = 4.740511491s iteration = 1 status = {"changed_state":[],"disabled":false,"errors":[],"instances_found":{"Failed":0,"SagaUnwound":0},"instances_reincarnated":[],"restart_errors":[]}
11372025-04-30T06:43:46.980ZTRACtest_utilization (BackgroundTasks): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet
11382025-04-30T06:43:46.980ZTRACtest_utilization (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = v2p_manager resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
11392025-04-30T06:43:46.981ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database
11402025-04-30T06:43:46.981ZDEBGtest_utilization (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
11412025-04-30T06:43:46.982ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
11422025-04-30T06:43:46.982ZDEBGtest_utilization (BackgroundTasks): inventory_prune_one: nothing eligible for removal (too few) background_task = inventory_collection candidates = []
11432025-04-30T06:43:46.983ZTRACtest_utilization (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal
11442025-04-30T06:43:46.983ZDEBGtest_utilization (BackgroundTasks): activation complete background_task = region_snapshot_replacement_step elapsed = 4.734545423s iteration = 1 status = {"errors":[],"step_garbage_collect_invoked_ok":[],"step_invoked_ok":[],"step_records_created_ok":[],"step_set_volume_deleted_ok":[]}
11452025-04-30T06:43:46.991ZINFOtest_utilization (dropshot): accepted connection local_addr = [::1]:39365 remote_addr = [::1]:52979 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
11462025-04-30T06:43:46.991ZTRACtest_utilization (dropshot): incoming request local_addr = [::1]:39365 method = GET remote_addr = [::1]:52979 req_id = 8f98ed0e-8827-429a-926f-e32a2954f393 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /vpc-routes
11472025-04-30T06:43:46.993ZTRACtest_utilization (dns): buffer buffer = Length: 67 (0x43) bytes\n0000: 36 1f 01 00 00 01 00 00 00 00 00 01 04 5f 6d 67 6............_mg\n0010: 73 04 5f 74 63 70 0d 63 6f 6e 74 72 6f 6c 2d 70 s._tcp.control-p\n0020: 6c 61 6e 65 05 6f 78 69 64 65 08 69 6e 74 65 72 lane.oxide.inter\n0030: 6e 61 6c 00 00 21 00 01 00 00 29 04 d0 00 00 00 nal..!....).....\n0040: 00 00 00 ... kind = dns peer_addr = [::1]:59496 req_id = 1a38f531-3dc4-46c3-8f08-bc1e2898d9b6
11482025-04-30T06:43:46.993ZDEBGtest_utilization (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 13855,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_mgs._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_mgs._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 4,\n 95,\n 109,\n 103,\n 115,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:59496 req_id = 1a38f531-3dc4-46c3-8f08-bc1e2898d9b6
11492025-04-30T06:43:46.993ZDEBGtest_utilization (store): query key key = _mgs._tcp
11502025-04-30T06:43:46.993ZDEBGtest_utilization (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
11512025-04-30T06:43:46.993ZDEBGtest_utilization (dns): dns response additional_records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal"), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] kind = dns peer_addr = [::1]:59496 query = LowerQuery { name: LowerName(Name("_mgs._tcp.control-plane.oxide.internal.")), original: Query { name: Name("_mgs._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN } } records = [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 48759, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal") })) }] req_id = 1a38f531-3dc4-46c3-8f08-bc1e2898d9b6
11522025-04-30T06:43:46.994ZINFOtest_utilization (dropshot): request completed latency_us = 2574 local_addr = [::1]:39365 method = GET remote_addr = [::1]:52979 req_id = 8f98ed0e-8827-429a-926f-e32a2954f393 response_code = 200 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /vpc-routes
11532025-04-30T06:43:46.994ZDEBGtest_utilization (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgs._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 48759, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }], valid_until: Instant { tv_sec: 4291, tv_nsec: 793753245 } })
11542025-04-30T06:43:46.994ZTRACtest_utilization (DnsResolver): lookup_service_targets: looking up SRV target
11552025-04-30T06:43:46.995ZDEBGtest_utilization (BackgroundTasks): client response SledAgent = b6d65341-167c-41df-9b5c-41cded99c229 background_task = vpc_route_manager result = Ok(Response { url: "http://[::1]:39365/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "8f98ed0e-8827-429a-926f-e32a2954f393", "content-length": "2", "date": "Wed, 30 Apr 2025 06:43:46 GMT"} })
11562025-04-30T06:43:46.995ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database
11572025-04-30T06:43:46.995ZDEBGtest_utilization (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
11582025-04-30T06:43:46.996ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
11592025-04-30T06:43:46.997ZTRACtest_utilization (dns): buffer buffer = Length: 108 (0x6c) bytes\n0000: 6b cc 01 00 00 01 00 00 00 00 00 01 2d 64 65 6e k...........-den\n0010: 64 72 69 74 65 2d 62 36 64 36 35 33 34 31 2d 31 drite-b6d65341-1\n0020: 36 37 63 2d 34 31 64 66 2d 39 62 35 63 2d 34 31 67c-41df-9b5c-41\n0030: 63 64 65 64 39 39 63 32 32 39 04 68 6f 73 74 0d cded99c229.host.\n0040: 63 6f 6e 74 72 6f 6c 2d 70 6c 61 6e 65 05 6f 78 control-plane.ox\n0050: 69 64 65 08 69 6e 74 65 72 6e 61 6c 00 00 1c 00 ide.internal....\n0060: 01 00 00 29 04 d0 00 00 00 00 00 00 ...)........ kind = dns peer_addr = [::1]:49496 req_id = b475493f-e732-4f9a-a63c-499875c04b32
11602025-04-30T06:43:46.997ZDEBGtest_utilization (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 27596,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n query_type: AAAA,\n query_class: IN,\n },\n },\n original: [\n 45,\n 100,\n 101,\n 110,\n 100,\n 114,\n 105,\n 116,\n 101,\n 45,\n 98,\n 54,\n 100,\n 54,\n 53,\n 51,\n 52,\n 49,\n 45,\n 49,\n 54,\n 55,\n 99,\n 45,\n 52,\n 49,\n 100,\n 102,\n 45,\n 57,\n 98,\n 53,\n 99,\n 45,\n 52,\n 49,\n 99,\n 100,\n 101,\n 100,\n 57,\n 57,\n 99,\n 50,\n 50,\n 57,\n 4,\n 104,\n 111,\n 115,\n 116,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 28,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:49496 req_id = b475493f-e732-4f9a-a63c-499875c04b32
11612025-04-30T06:43:46.997ZDEBGtest_utilization (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
11622025-04-30T06:43:46.997ZDEBGtest_utilization (dns): dns response additional_records = [] kind = dns peer_addr = [::1]:49496 query = LowerQuery { name: LowerName(Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.")), original: Query { name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), query_type: AAAA, query_class: IN } } records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] req_id = b475493f-e732-4f9a-a63c-499875c04b32
11632025-04-30T06:43:47.081ZTRACtest_utilization (DnsResolver): lookup_all_socket_v6 srv dns_name = _clickhouse-admin-keeper._tcp.control-plane.oxide.internal
11642025-04-30T06:43:47.082ZTRACtest_utilization (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = service_firewall_rule_propagation resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
11652025-04-30T06:43:47.082ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database
11662025-04-30T06:43:47.083ZDEBGtest_utilization (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
11672025-04-30T06:43:47.084ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
11682025-04-30T06:43:47.084ZINFOtest_utilization (InternalApi): set cockroachdb setting setting = cluster.preserve_downgrade_option value = 22.1
11692025-04-30T06:43:47.084ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
11702025-04-30T06:43:47.084ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
11712025-04-30T06:43:47.085ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
11722025-04-30T06:43:47.089ZTRACtest_utilization (dns): buffer buffer = Length: 87 (0x57) bytes\n0000: 3f 36 01 00 00 01 00 00 00 00 00 01 18 5f 63 6c ?6..........._cl\n0010: 69 63 6b 68 6f 75 73 65 2d 61 64 6d 69 6e 2d 6b ickhouse-admin-k\n0020: 65 65 70 65 72 04 5f 74 63 70 0d 63 6f 6e 74 72 eeper._tcp.contr\n0030: 6f 6c 2d 70 6c 61 6e 65 05 6f 78 69 64 65 08 69 ol-plane.oxide.i\n0040: 6e 74 65 72 6e 61 6c 00 00 21 00 01 00 00 29 04 nternal..!....).\n0050: d0 00 00 00 00 00 00 ....... kind = dns peer_addr = [::1]:64728 req_id = 97c5646e-be8a-4cd8-9f67-07bf33d564a6
11732025-04-30T06:43:47.090ZDEBGtest_utilization (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 16182,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 24,\n 95,\n 99,\n 108,\n 105,\n 99,\n 107,\n 104,\n 111,\n 117,\n 115,\n 101,\n 45,\n 97,\n 100,\n 109,\n 105,\n 110,\n 45,\n 107,\n 101,\n 101,\n 112,\n 101,\n 114,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:64728 req_id = 97c5646e-be8a-4cd8-9f67-07bf33d564a6
11742025-04-30T06:43:47.090ZDEBGtest_utilization (store): query key key = _clickhouse-admin-keeper._tcp
11752025-04-30T06:43:47.090ZERROtest_utilization (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 16182, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 1, }, query: WireQuery { query: LowerQuery { name: LowerName( Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), ), original: Query { name: Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN, }, }, original: [ 24, 95, 99, 108, 105, 99, 107, 104, 111, 117, 115, 101, 45, 97, 100, 109, 105, 110, 45, 107, 101, 101, 112, 101, 114, 4, 95, 116, 99, 112, 13, 99, 111, 110, 116, 114, 111, 108, 45, 112, 108, 97, 110, 101, 5, 111, 120, 105, 100, 101, 8, 105, 110, 116, 101, 114, 110, 97, 108, 0, 0, 33, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: Some( Edns { rcode_high: 0, version: 0, dnssec_ok: false, max_payload: 1232, options: OPT { options: {}, }, }, ), } NXDOMAIN: no records found for name: "_clickhouse-admin-keeper._tcp.control-plane.oxide.internal." kind = dns peer_addr = [::1]:64728 req_id = 97c5646e-be8a-4cd8-9f67-07bf33d564a6
11762025-04-30T06:43:47.090ZDEBGtest_utilization (BackgroundTasks): roles background_task = tuf_artifact_replication roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
11772025-04-30T06:43:47.095ZDEBGtest_utilization (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = tuf_artifact_replication resource = Fleet result = Ok(())
11782025-04-30T06:43:47.095ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = tuf_artifact_replication resource = Database
11792025-04-30T06:43:47.095ZDEBGtest_utilization (BackgroundTasks): roles background_task = tuf_artifact_replication roles = RoleSet { roles: {} }
11802025-04-30T06:43:47.096ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = tuf_artifact_replication resource = Database result = Ok(())
11812025-04-30T06:43:47.096ZDEBGtest_utilization (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
11822025-04-30T06:43:47.102ZDEBGtest_utilization (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
11832025-04-30T06:43:47.102ZTRACtest_utilization (BackgroundTasks): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet
11842025-04-30T06:43:47.102ZTRACtest_utilization (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = v2p_manager resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
11852025-04-30T06:43:47.102ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database
11862025-04-30T06:43:47.102ZDEBGtest_utilization (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
11872025-04-30T06:43:47.103ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
11882025-04-30T06:43:47.112ZDEBGtest_utilization (BackgroundTasks): begin collection background_task = inventory_collection
11892025-04-30T06:43:47.112ZDEBGtest_utilization (BackgroundTasks): begin collection from MGS background_task = inventory_collection mgs_url = http://[::1]:48759
11902025-04-30T06:43:47.112ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/ignition
11912025-04-30T06:43:47.121ZDEBGtest_utilization (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
11922025-04-30T06:43:47.149ZDEBGtest_utilization (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
11932025-04-30T06:43:47.149ZTRACtest_utilization (BackgroundTasks): successful lookup of siloed resource "Vpc" using built-in user background_task = service_firewall_rule_propagation
11942025-04-30T06:43:47.149ZTRACtest_utilization (BackgroundTasks): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) }
11952025-04-30T06:43:47.149ZTRACtest_utilization (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = service_firewall_rule_propagation resource_id = 001de000-4401-4000-8000-000000000000 resource_type = Project
11962025-04-30T06:43:47.149ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database
11972025-04-30T06:43:47.149ZDEBGtest_utilization (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
11982025-04-30T06:43:47.149ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
11992025-04-30T06:43:47.156ZTRACtest_utilization (BackgroundTasks): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = tuf_artifact_replication resource = Fleet
12002025-04-30T06:43:47.156ZTRACtest_utilization (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = tuf_artifact_replication resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
12012025-04-30T06:43:47.156ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = tuf_artifact_replication resource = Database
12022025-04-30T06:43:47.156ZDEBGtest_utilization (BackgroundTasks): roles background_task = tuf_artifact_replication roles = RoleSet { roles: {} }
12032025-04-30T06:43:47.157ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = tuf_artifact_replication resource = Database result = Ok(())
12042025-04-30T06:43:47.157ZDEBGtest_utilization (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
12052025-04-30T06:43:47.160ZDEBGtest_utilization (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
12062025-04-30T06:43:47.160ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database
12072025-04-30T06:43:47.160ZDEBGtest_utilization (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
12082025-04-30T06:43:47.161ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
12092025-04-30T06:43:47.167ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/ignition", status: 200, headers: {"content-type": "application/json", "x-request-id": "d661edaf-ff43-4261-ae4a-d196e858304c", "content-length": "835", "date": "Wed, 30 Apr 2025 06:43:46 GMT"} })
12102025-04-30T06:43:47.168ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0
12112025-04-30T06:43:47.213ZTRACtest_utilization (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = service_firewall_rule_propagation resource_id = 001de000-5110-4000-8000-000000000001 resource_type = Silo
12122025-04-30T06:43:47.213ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database
12132025-04-30T06:43:47.213ZDEBGtest_utilization (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
12142025-04-30T06:43:47.214ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
12152025-04-30T06:43:47.214ZTRACtest_utilization (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Rack { parent: Fleet, key: c19a698f-c6f9-4a17-ae30-20d711b8f7dc, lookup_type: ById(c19a698f-c6f9-4a17-ae30-20d711b8f7dc) }
12162025-04-30T06:43:47.215ZTRACtest_utilization (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
12172025-04-30T06:43:47.215ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
12182025-04-30T06:43:47.216ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
12192025-04-30T06:43:47.216ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
12202025-04-30T06:43:47.230ZDEBGtest_utilization (BackgroundTasks): roles background_task = tuf_artifact_replication roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
12212025-04-30T06:43:47.241ZDEBGtest_utilization (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = tuf_artifact_replication resource = Fleet result = Ok(())
12222025-04-30T06:43:47.241ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = tuf_artifact_replication resource = Database
12232025-04-30T06:43:47.241ZDEBGtest_utilization (BackgroundTasks): roles background_task = tuf_artifact_replication roles = RoleSet { roles: {} }
12242025-04-30T06:43:47.243ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = tuf_artifact_replication resource = Database result = Ok(())
12252025-04-30T06:43:47.312ZDEBGtest_utilization (BackgroundTasks): client request SledAgent = b6d65341-167c-41df-9b5c-41cded99c229 background_task = v2p_manager body = None method = GET uri = http://[::1]:39365/v2p
12262025-04-30T06:43:47.312ZTRACtest_utilization (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = service_firewall_rule_propagation resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
12272025-04-30T06:43:47.312ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database
12282025-04-30T06:43:47.312ZDEBGtest_utilization (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
12292025-04-30T06:43:47.316ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
12302025-04-30T06:43:47.317ZINFOtest_utilization (dropshot): accepted connection local_addr = [::1]:39365 remote_addr = [::1]:45188 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
12312025-04-30T06:43:47.320ZTRACtest_utilization (dropshot): incoming request local_addr = [::1]:39365 method = GET remote_addr = [::1]:45188 req_id = 58390ab8-e7b4-4db4-8906-50bc348bd4c4 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /v2p
12322025-04-30T06:43:47.320ZINFOtest_utilization (dropshot): request completed latency_us = 632 local_addr = [::1]:39365 method = GET remote_addr = [::1]:45188 req_id = 58390ab8-e7b4-4db4-8906-50bc348bd4c4 response_code = 200 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /v2p
12332025-04-30T06:43:47.325ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "a684a944-9b20-4c5e-9961-4f58107d720d", "content-length": "734", "date": "Wed, 30 Apr 2025 06:43:46 GMT"} })
12342025-04-30T06:43:47.326ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0/component/sp/caboose?firmware_slot=0
12352025-04-30T06:43:47.326ZDEBGtest_utilization (BackgroundTasks): client response SledAgent = b6d65341-167c-41df-9b5c-41cded99c229 background_task = v2p_manager result = Ok(Response { url: "http://[::1]:39365/v2p", status: 200, headers: {"content-type": "application/json", "x-request-id": "58390ab8-e7b4-4db4-8906-50bc348bd4c4", "content-length": "2", "date": "Wed, 30 Apr 2025 06:43:46 GMT"} })
12362025-04-30T06:43:47.337ZINFOtest_utilization (BackgroundTasks): found opte v2p mappings background_task = v2p_manager interfaces = {} sled = sim-b6d65341
12372025-04-30T06:43:47.337ZINFOtest_utilization (BackgroundTasks): v2p mappings to delete background_task = v2p_manager mappings = [] sled = sim-b6d65341
12382025-04-30T06:43:47.337ZINFOtest_utilization (BackgroundTasks): v2p mappings to add background_task = v2p_manager mappings = [] sled = sim-b6d65341
12392025-04-30T06:43:47.339ZDEBGtest_utilization (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 5.103946711s iteration = 1 status = {}
12402025-04-30T06:43:47.340ZINFOtest_utilization (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager sled = sim-b6d65341
12412025-04-30T06:43:47.341ZDEBGtest_utilization (BackgroundTasks): client request SledAgent = b6d65341-167c-41df-9b5c-41cded99c229 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[::1]:39365/eip-gateways
12422025-04-30T06:43:47.341ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
12432025-04-30T06:43:47.349ZDEBGtest_utilization (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Rack { parent: Fleet, key: c19a698f-c6f9-4a17-ae30-20d711b8f7dc, lookup_type: ById(c19a698f-c6f9-4a17-ae30-20d711b8f7dc) } result = Ok(())
12442025-04-30T06:43:47.349ZDEBGtest_utilization (InternalApi): updating rack subnet for rack c19a698f-c6f9-4a17-ae30-20d711b8f7dc to Some( V6( Ipv6Network { addr: fd00:1122:3344:100::, prefix: 56, }, ), )
12452025-04-30T06:43:47.349ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
12462025-04-30T06:43:47.349ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
12472025-04-30T06:43:47.350ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
12482025-04-30T06:43:47.354ZTRACtest_utilization (dropshot): incoming request local_addr = [::1]:39365 method = PUT remote_addr = [::1]:52979 req_id = a9bdacaf-8b35-4db0-84d1-fc7f0813bf3d sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /eip-gateways
12492025-04-30T06:43:47.357ZINFOtest_utilization (dropshot): request completed latency_us = 2371 local_addr = [::1]:39365 method = PUT remote_addr = [::1]:52979 req_id = a9bdacaf-8b35-4db0-84d1-fc7f0813bf3d response_code = 204 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /eip-gateways
12502025-04-30T06:43:47.363ZDEBGtest_utilization (BackgroundTasks): client request SledAgent = b6d65341-167c-41df-9b5c-41cded99c229 background_task = tuf_artifact_replication body = Some(Body) method = PUT uri = http://[::1]:39365/artifacts-config
12512025-04-30T06:43:47.364ZDEBGtest_utilization (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
12522025-04-30T06:43:47.371ZDEBGtest_utilization (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
12532025-04-30T06:43:47.371ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database
12542025-04-30T06:43:47.371ZDEBGtest_utilization (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
12552025-04-30T06:43:47.372ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
12562025-04-30T06:43:47.372ZINFOtest_utilization (dropshot): accepted connection local_addr = [::1]:39365 remote_addr = [::1]:64506 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
12572025-04-30T06:43:47.374ZDEBGtest_utilization (BackgroundTasks): client response SledAgent = b6d65341-167c-41df-9b5c-41cded99c229 background_task = vpc_route_manager result = Ok(Response { url: "http://[::1]:39365/eip-gateways", status: 204, headers: {"x-request-id": "a9bdacaf-8b35-4db0-84d1-fc7f0813bf3d", "date": "Wed, 30 Apr 2025 06:43:46 GMT"} })
12582025-04-30T06:43:47.375ZDEBGtest_utilization (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 5.132997032s iteration = 1 status = {}
12592025-04-30T06:43:47.379ZTRACtest_utilization (dropshot): incoming request local_addr = [::1]:39365 method = PUT remote_addr = [::1]:64506 req_id = fef081ee-ada9-4acc-9202-6641af72a6d2 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /artifacts-config
12602025-04-30T06:43:47.381ZINFOtest_utilization (ArtifactStore): No ledger in /var/tmp/omicron_tmp/.tmp5QxCob/artifact-config.json server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
12612025-04-30T06:43:47.381ZDEBGtest_utilization (ArtifactStore): Failed to read ledger: Not found in storage path = /var/tmp/omicron_tmp/.tmp5QxCob/artifact-config.json server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
12622025-04-30T06:43:47.381ZINFOtest_utilization (ArtifactStore): No ledger in /var/tmp/omicron_tmp/.tmpRU082q/artifact-config.json server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
12632025-04-30T06:43:47.381ZDEBGtest_utilization (ArtifactStore): Failed to read ledger: Not found in storage path = /var/tmp/omicron_tmp/.tmpRU082q/artifact-config.json server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
12642025-04-30T06:43:47.382ZINFOtest_utilization (ArtifactStore): Writing ledger to /var/tmp/omicron_tmp/.tmp5QxCob/.artifact-config.json.tmp server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
12652025-04-30T06:43:47.383ZINFOtest_utilization (ArtifactStore): Writing ledger to /var/tmp/omicron_tmp/.tmpRU082q/.artifact-config.json.tmp server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
12662025-04-30T06:43:47.384ZINFOtest_utilization (ArtifactStore): Starting delete reconciler generation = 1 server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
12672025-04-30T06:43:47.384ZINFOtest_utilization (dropshot): request completed latency_us = 5064 local_addr = [::1]:39365 method = PUT remote_addr = [::1]:64506 req_id = fef081ee-ada9-4acc-9202-6641af72a6d2 response_code = 204 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /artifacts-config
12682025-04-30T06:43:47.385ZDEBGtest_utilization (BackgroundTasks): client response SledAgent = b6d65341-167c-41df-9b5c-41cded99c229 background_task = tuf_artifact_replication result = Ok(Response { url: "http://[::1]:39365/artifacts-config", status: 204, headers: {"x-request-id": "fef081ee-ada9-4acc-9202-6641af72a6d2", "date": "Wed, 30 Apr 2025 06:43:46 GMT"} })
12692025-04-30T06:43:47.385ZDEBGtest_utilization (BackgroundTasks): client request SledAgent = b6d65341-167c-41df-9b5c-41cded99c229 background_task = tuf_artifact_replication body = None method = GET uri = http://[::1]:39365/artifacts
12702025-04-30T06:43:47.385ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database
12712025-04-30T06:43:47.385ZDEBGtest_utilization (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
12722025-04-30T06:43:47.387ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
12732025-04-30T06:43:47.387ZINFOtest_utilization (InternalApi): Using static configuration for external switchports
12742025-04-30T06:43:47.387ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
12752025-04-30T06:43:47.387ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
12762025-04-30T06:43:47.388ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
12772025-04-30T06:43:47.388ZTRACtest_utilization (dropshot): incoming request local_addr = [::1]:39365 method = GET remote_addr = [::1]:64506 req_id = 5df6c262-93fe-4cc5-8b38-e3008fee5d03 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /artifacts
12782025-04-30T06:43:47.389ZINFOtest_utilization (dropshot): request completed latency_us = 1295 local_addr = [::1]:39365 method = GET remote_addr = [::1]:64506 req_id = 5df6c262-93fe-4cc5-8b38-e3008fee5d03 response_code = 200 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /artifacts
12792025-04-30T06:43:47.390ZDEBGtest_utilization (BackgroundTasks): client response SledAgent = b6d65341-167c-41df-9b5c-41cded99c229 background_task = tuf_artifact_replication result = Ok(Response { url: "http://[::1]:39365/artifacts", status: 200, headers: {"content-type": "application/json", "x-request-id": "5df6c262-93fe-4cc5-8b38-e3008fee5d03", "content-length": "26", "date": "Wed, 30 Apr 2025 06:43:46 GMT"} })
12802025-04-30T06:43:47.391ZINFOtest_utilization (BackgroundTasks): Successfully got artifact list background_task = tuf_artifact_replication sled = http://[::1]:39365
12812025-04-30T06:43:47.393ZDEBGtest_utilization (BackgroundTasks): activation complete background_task = tuf_artifact_replication elapsed = 5.143373757s iteration = 1 status = {"generation":1,"last_run_counters":{"copy_err":0,"copy_ok":0,"list_err":0,"list_ok":1,"put_config_err":0,"put_config_ok":1,"put_err":0,"put_ok":0},"lifetime_counters":{"copy_err":0,"copy_ok":0,"list_err":0,"list_ok":1,"put_config_err":0,"put_config_ok":1,"put_err":0,"put_ok":0},"local_repos":0,"request_debug_ringbuf":[{"generation":1,"operation":"put_config","target_sled":"b6d65341-167c-41df-9b5c-41cded99c229","time":"2025-04-30T06:43:47.385318485Z"},{"operation":"list","target_sled":"b6d65341-167c-41df-9b5c-41cded99c229","time":"2025-04-30T06:43:47.391165264Z"}]}
12822025-04-30T06:43:47.393ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database
12832025-04-30T06:43:47.393ZDEBGtest_utilization (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
12842025-04-30T06:43:47.394ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
12852025-04-30T06:43:47.396ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "4100c631-5250-4896-a800-d9e4ccf3b500", "content-length": "111", "date": "Wed, 30 Apr 2025 06:43:46 GMT"} })
12862025-04-30T06:43:47.398ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0/component/sp/caboose?firmware_slot=1
12872025-04-30T06:43:47.400ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "172c737c-8ef4-42d4-827b-e86b3e53f8b9", "content-length": "111", "date": "Wed, 30 Apr 2025 06:43:46 GMT"} })
12882025-04-30T06:43:47.401ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0/component/rot/caboose?firmware_slot=0
12892025-04-30T06:43:47.405ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "e5fb2551-2335-43b1-a07f-ff687bb5752a", "content-length": "170", "date": "Wed, 30 Apr 2025 06:43:46 GMT"} })
12902025-04-30T06:43:47.406ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0/component/rot/caboose?firmware_slot=1
12912025-04-30T06:43:47.446ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database
12922025-04-30T06:43:47.446ZDEBGtest_utilization (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
12932025-04-30T06:43:47.453ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
12942025-04-30T06:43:47.463ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "e0ad2f07-857f-44c5-8218-3d4a82cb767a", "content-length": "170", "date": "Wed, 30 Apr 2025 06:43:46 GMT"} })
12952025-04-30T06:43:47.464ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0/component/stage0/caboose?firmware_slot=0
12962025-04-30T06:43:47.464ZTRACtest_utilization (BackgroundTasks): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) }
12972025-04-30T06:43:47.464ZTRACtest_utilization (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = service_firewall_rule_propagation resource_id = 001de000-4401-4000-8000-000000000000 resource_type = Project
12982025-04-30T06:43:47.464ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database
12992025-04-30T06:43:47.464ZDEBGtest_utilization (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
13002025-04-30T06:43:47.464ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
13012025-04-30T06:43:47.467ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "ee7a839b-4347-4c23-82b2-5bf61994db08", "content-length": "179", "date": "Wed, 30 Apr 2025 06:43:46 GMT"} })
13022025-04-30T06:43:47.468ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0/component/stage0/caboose?firmware_slot=1
13032025-04-30T06:43:47.479ZTRACtest_utilization (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = service_firewall_rule_propagation resource_id = 001de000-5110-4000-8000-000000000001 resource_type = Silo
13042025-04-30T06:43:47.479ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database
13052025-04-30T06:43:47.479ZDEBGtest_utilization (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
13062025-04-30T06:43:47.480ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
13072025-04-30T06:43:47.485ZINFOtest_utilization (InternalApi): Recording Rack Network Configuration
13082025-04-30T06:43:47.485ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
13092025-04-30T06:43:47.485ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
13102025-04-30T06:43:47.485ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
13112025-04-30T06:43:47.486ZTRACtest_utilization (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = service_firewall_rule_propagation resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
13122025-04-30T06:43:47.486ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database
13132025-04-30T06:43:47.486ZDEBGtest_utilization (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
13142025-04-30T06:43:47.486ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
13152025-04-30T06:43:47.489ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "85ddc886-19b6-40ee-9297-c7915960c16e", "content-length": "179", "date": "Wed, 30 Apr 2025 06:43:47 GMT"} })
13162025-04-30T06:43:47.489ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0/component/rot/cmpa
13172025-04-30T06:43:47.489ZDEBGtest_utilization (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
13182025-04-30T06:43:47.499ZDEBGtest_utilization (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
13192025-04-30T06:43:47.499ZTRACtest_utilization (BackgroundTasks): successful lookup of siloed resource "Vpc" using built-in user background_task = service_firewall_rule_propagation
13202025-04-30T06:43:47.499ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database
13212025-04-30T06:43:47.499ZDEBGtest_utilization (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
13222025-04-30T06:43:47.501ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
13232025-04-30T06:43:47.525ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database
13242025-04-30T06:43:47.525ZDEBGtest_utilization (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
13252025-04-30T06:43:47.525ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
13262025-04-30T06:43:47.527ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "28de5dd7-b976-4e40-9604-28b3041a3a4a", "content-length": "702", "date": "Wed, 30 Apr 2025 06:43:47 GMT"} })
13272025-04-30T06:43:47.527ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0/component/rot/cfpa
13282025-04-30T06:43:47.529ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database
13292025-04-30T06:43:47.529ZDEBGtest_utilization (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
13302025-04-30T06:43:47.530ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
13312025-04-30T06:43:47.531ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "043fb005-b6cd-42e7-b79d-0f2774675df0", "content-length": "727", "date": "Wed, 30 Apr 2025 06:43:47 GMT"} })
13322025-04-30T06:43:47.531ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0/component/rot/cfpa
13332025-04-30T06:43:47.533ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "77d1be8c-42a0-409b-98f4-ec547cfce6c4", "content-length": "729", "date": "Wed, 30 Apr 2025 06:43:47 GMT"} })
13342025-04-30T06:43:47.533ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0/component/rot/cfpa
13352025-04-30T06:43:47.534ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database
13362025-04-30T06:43:47.534ZDEBGtest_utilization (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
13372025-04-30T06:43:47.535ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
13382025-04-30T06:43:47.535ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "e0181643-e632-4ccf-82cd-094c30939d77", "content-length": "728", "date": "Wed, 30 Apr 2025 06:43:47 GMT"} })
13392025-04-30T06:43:47.535ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1
13402025-04-30T06:43:47.538ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1", status: 200, headers: {"content-type": "application/json", "x-request-id": "d8ae8fcd-68b1-42fa-be80-02ff0ee6c056", "content-length": "734", "date": "Wed, 30 Apr 2025 06:43:47 GMT"} })
13412025-04-30T06:43:47.538ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1/component/sp/caboose?firmware_slot=0
13422025-04-30T06:43:47.538ZTRACtest_utilization (BackgroundTasks): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000001, lookup_type: ByName("external-dns") }
13432025-04-30T06:43:47.538ZTRACtest_utilization (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = service_firewall_rule_propagation resource_id = 001de000-4401-4000-8000-000000000000 resource_type = Project
13442025-04-30T06:43:47.538ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database
13452025-04-30T06:43:47.538ZDEBGtest_utilization (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
13462025-04-30T06:43:47.539ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
13472025-04-30T06:43:47.544ZTRACtest_utilization (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = service_firewall_rule_propagation resource_id = 001de000-5110-4000-8000-000000000001 resource_type = Silo
13482025-04-30T06:43:47.544ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database
13492025-04-30T06:43:47.544ZDEBGtest_utilization (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
13502025-04-30T06:43:47.545ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
13512025-04-30T06:43:47.545ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "b70ac27f-51bf-419b-a5c9-1e90bbbd32f8", "content-length": "111", "date": "Wed, 30 Apr 2025 06:43:47 GMT"} })
13522025-04-30T06:43:47.546ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1/component/sp/caboose?firmware_slot=1
13532025-04-30T06:43:47.548ZTRACtest_utilization (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = service_firewall_rule_propagation resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
13542025-04-30T06:43:47.548ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database
13552025-04-30T06:43:47.548ZDEBGtest_utilization (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
13562025-04-30T06:43:47.549ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
13572025-04-30T06:43:47.551ZDEBGtest_utilization (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
13582025-04-30T06:43:47.560ZDEBGtest_utilization (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000001, lookup_type: ByName("external-dns") } result = Ok(())
13592025-04-30T06:43:47.560ZTRACtest_utilization (BackgroundTasks): successful lookup of siloed resource "VpcSubnet" using built-in user background_task = service_firewall_rule_propagation
13602025-04-30T06:43:47.560ZTRACtest_utilization (BackgroundTasks): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000001, lookup_type: ByName("external-dns") }
13612025-04-30T06:43:47.560ZTRACtest_utilization (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = service_firewall_rule_propagation resource_id = 001de000-4401-4000-8000-000000000000 resource_type = Project
13622025-04-30T06:43:47.560ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database
13632025-04-30T06:43:47.560ZDEBGtest_utilization (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
13642025-04-30T06:43:47.561ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
13652025-04-30T06:43:47.565ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "29042f9d-bd29-49a8-a0b0-a39da3c45e7e", "content-length": "111", "date": "Wed, 30 Apr 2025 06:43:47 GMT"} })
13662025-04-30T06:43:47.566ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1/component/rot/caboose?firmware_slot=0
13672025-04-30T06:43:47.566ZTRACtest_utilization (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = service_firewall_rule_propagation resource_id = 001de000-5110-4000-8000-000000000001 resource_type = Silo
13682025-04-30T06:43:47.566ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database
13692025-04-30T06:43:47.566ZDEBGtest_utilization (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
13702025-04-30T06:43:47.567ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
13712025-04-30T06:43:47.571ZTRACtest_utilization (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = service_firewall_rule_propagation resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
13722025-04-30T06:43:47.571ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database
13732025-04-30T06:43:47.571ZDEBGtest_utilization (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
13742025-04-30T06:43:47.572ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
13752025-04-30T06:43:47.590ZDEBGtest_utilization (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
13762025-04-30T06:43:47.612ZDEBGtest_utilization (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000001, lookup_type: ByName("external-dns") } result = Ok(())
13772025-04-30T06:43:47.612ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database
13782025-04-30T06:43:47.612ZDEBGtest_utilization (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
13792025-04-30T06:43:47.613ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
13802025-04-30T06:43:47.663ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database
13812025-04-30T06:43:47.663ZDEBGtest_utilization (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
13822025-04-30T06:43:47.665ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
13832025-04-30T06:43:47.675ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database
13842025-04-30T06:43:47.676ZDEBGtest_utilization (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
13852025-04-30T06:43:47.676ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
13862025-04-30T06:43:47.701ZTRACtest_utilization (InternalApi): authorize begin action = CreateChild actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Fleet
13872025-04-30T06:43:47.701ZTRACtest_utilization (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
13882025-04-30T06:43:47.701ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
13892025-04-30T06:43:47.701ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
13902025-04-30T06:43:47.702ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
13912025-04-30T06:43:47.702ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database
13922025-04-30T06:43:47.702ZDEBGtest_utilization (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
13932025-04-30T06:43:47.703ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
13942025-04-30T06:43:47.722ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database
13952025-04-30T06:43:47.722ZDEBGtest_utilization (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
13962025-04-30T06:43:47.723ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
13972025-04-30T06:43:47.724ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
13982025-04-30T06:43:47.733ZDEBGtest_utilization (InternalApi): authorize result action = CreateChild actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Fleet result = Ok(())
13992025-04-30T06:43:47.733ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
14002025-04-30T06:43:47.733ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
14012025-04-30T06:43:47.734ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
14022025-04-30T06:43:47.747ZTRACtest_utilization (BackgroundTasks): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000002, lookup_type: ByName("nexus") }
14032025-04-30T06:43:47.748ZTRACtest_utilization (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = service_firewall_rule_propagation resource_id = 001de000-4401-4000-8000-000000000000 resource_type = Project
14042025-04-30T06:43:47.748ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database
14052025-04-30T06:43:47.748ZDEBGtest_utilization (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
14062025-04-30T06:43:47.748ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
14072025-04-30T06:43:47.750ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "2d6ffe15-5aeb-46c1-9715-220138cb7102", "content-length": "170", "date": "Wed, 30 Apr 2025 06:43:47 GMT"} })
14082025-04-30T06:43:47.750ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1/component/rot/caboose?firmware_slot=1
14092025-04-30T06:43:47.751ZTRACtest_utilization (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = IpPool { parent: Fleet, key: ffe9c0eb-aaed-4d7f-82cb-41bf985bfb09, lookup_type: ByName("oxide-service-pool") }
14102025-04-30T06:43:47.751ZTRACtest_utilization (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
14112025-04-30T06:43:47.751ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
14122025-04-30T06:43:47.751ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
14132025-04-30T06:43:47.752ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
14142025-04-30T06:43:47.755ZTRACtest_utilization (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = service_firewall_rule_propagation resource_id = 001de000-5110-4000-8000-000000000001 resource_type = Silo
14152025-04-30T06:43:47.755ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database
14162025-04-30T06:43:47.755ZDEBGtest_utilization (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
14172025-04-30T06:43:47.767ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
14182025-04-30T06:43:47.768ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
14192025-04-30T06:43:47.775ZDEBGtest_utilization (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = IpPool { parent: Fleet, key: ffe9c0eb-aaed-4d7f-82cb-41bf985bfb09, lookup_type: ByName("oxide-service-pool") } result = Ok(())
14202025-04-30T06:43:47.776ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
14212025-04-30T06:43:47.776ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
14222025-04-30T06:43:47.776ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
14232025-04-30T06:43:47.776ZTRACtest_utilization (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = service_firewall_rule_propagation resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
14242025-04-30T06:43:47.776ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database
14252025-04-30T06:43:47.776ZDEBGtest_utilization (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
14262025-04-30T06:43:47.777ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
14272025-04-30T06:43:47.779ZDEBGtest_utilization (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
14282025-04-30T06:43:47.787ZDEBGtest_utilization (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000002, lookup_type: ByName("nexus") } result = Ok(())
14292025-04-30T06:43:47.787ZTRACtest_utilization (BackgroundTasks): successful lookup of siloed resource "VpcSubnet" using built-in user background_task = service_firewall_rule_propagation
14302025-04-30T06:43:47.787ZTRACtest_utilization (BackgroundTasks): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000002, lookup_type: ByName("nexus") }
14312025-04-30T06:43:47.787ZTRACtest_utilization (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = service_firewall_rule_propagation resource_id = 001de000-4401-4000-8000-000000000000 resource_type = Project
14322025-04-30T06:43:47.787ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database
14332025-04-30T06:43:47.787ZDEBGtest_utilization (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
14342025-04-30T06:43:47.788ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
14352025-04-30T06:43:47.788ZTRACtest_utilization (InternalApi): authorize begin action = CreateChild actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = IpPool { parent: Fleet, key: ffe9c0eb-aaed-4d7f-82cb-41bf985bfb09, lookup_type: ByName("oxide-service-pool") }
14362025-04-30T06:43:47.789ZTRACtest_utilization (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
14372025-04-30T06:43:47.789ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
14382025-04-30T06:43:47.789ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
14392025-04-30T06:43:47.790ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
14402025-04-30T06:43:47.791ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "fe18c671-85dd-4fef-b2f8-60f123c49f34", "content-length": "170", "date": "Wed, 30 Apr 2025 06:43:47 GMT"} })
14412025-04-30T06:43:47.791ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1/component/stage0/caboose?firmware_slot=0
14422025-04-30T06:43:47.792ZTRACtest_utilization (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = service_firewall_rule_propagation resource_id = 001de000-5110-4000-8000-000000000001 resource_type = Silo
14432025-04-30T06:43:47.792ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database
14442025-04-30T06:43:47.792ZDEBGtest_utilization (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
14452025-04-30T06:43:47.793ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
14462025-04-30T06:43:47.793ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
14472025-04-30T06:43:47.800ZDEBGtest_utilization (InternalApi): authorize result action = CreateChild actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = IpPool { parent: Fleet, key: ffe9c0eb-aaed-4d7f-82cb-41bf985bfb09, lookup_type: ByName("oxide-service-pool") } result = Ok(())
14482025-04-30T06:43:47.806ZTRACtest_utilization (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = service_firewall_rule_propagation resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
14492025-04-30T06:43:47.806ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database
14502025-04-30T06:43:47.806ZDEBGtest_utilization (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
14512025-04-30T06:43:47.807ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
14522025-04-30T06:43:47.809ZDEBGtest_utilization (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
14532025-04-30T06:43:47.845ZDEBGtest_utilization (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000002, lookup_type: ByName("nexus") } result = Ok(())
14542025-04-30T06:43:47.845ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database
14552025-04-30T06:43:47.845ZDEBGtest_utilization (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
14562025-04-30T06:43:47.846ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
14572025-04-30T06:43:47.846ZTRACtest_utilization (InternalApi): authorize begin action = CreateChild actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = IpPool { parent: Fleet, key: ffe9c0eb-aaed-4d7f-82cb-41bf985bfb09, lookup_type: ByName("oxide-service-pool") }
14582025-04-30T06:43:47.846ZTRACtest_utilization (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
14592025-04-30T06:43:47.847ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
14602025-04-30T06:43:47.847ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
14612025-04-30T06:43:47.847ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
14622025-04-30T06:43:47.847ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "e5ccf504-c832-42b9-9677-12c5f2f15f8f", "content-length": "179", "date": "Wed, 30 Apr 2025 06:43:47 GMT"} })
14632025-04-30T06:43:47.847ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1/component/stage0/caboose?firmware_slot=1
14642025-04-30T06:43:47.850ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
14652025-04-30T06:43:47.857ZDEBGtest_utilization (InternalApi): authorize result action = CreateChild actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = IpPool { parent: Fleet, key: ffe9c0eb-aaed-4d7f-82cb-41bf985bfb09, lookup_type: ByName("oxide-service-pool") } result = Ok(())
14662025-04-30T06:43:47.862ZDEBGtest_utilization (service-firewall-rule-progator): resolved names for firewall rules background_task = service_firewall_rule_propagation instance_interfaces = {} subnet_interfaces = {} subnet_networks = {Name("external-dns"): [V4(Ipv4Network { addr: 172.30.1.0, prefix: 24 }), V6(Ipv6Network { addr: fd77:e9d2:9cd9:1::, prefix: 64 })], Name("nexus"): [V4(Ipv4Network { addr: 172.30.2.0, prefix: 24 }), V6(Ipv6Network { addr: fd77:e9d2:9cd9:2::, prefix: 64 })]} vpc_interfaces = {}
14672025-04-30T06:43:47.862ZTRACtest_utilization (BackgroundTasks): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Fleet
14682025-04-30T06:43:47.862ZTRACtest_utilization (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = service_firewall_rule_propagation resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
14692025-04-30T06:43:47.862ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database
14702025-04-30T06:43:47.862ZDEBGtest_utilization (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
14712025-04-30T06:43:47.862ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
14722025-04-30T06:43:47.865ZTRACtest_utilization (InternalApi): authorize begin action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = BlueprintConfig
14732025-04-30T06:43:47.865ZTRACtest_utilization (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
14742025-04-30T06:43:47.865ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
14752025-04-30T06:43:47.865ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
14762025-04-30T06:43:47.865ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
14772025-04-30T06:43:47.866ZDEBGtest_utilization (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
14782025-04-30T06:43:47.872ZDEBGtest_utilization (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Fleet result = Ok(())
14792025-04-30T06:43:47.872ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database
14802025-04-30T06:43:47.872ZDEBGtest_utilization (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
14812025-04-30T06:43:47.873ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
14822025-04-30T06:43:47.875ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
14832025-04-30T06:43:47.885ZDEBGtest_utilization (InternalApi): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = BlueprintConfig result = Ok(())
14842025-04-30T06:43:47.895ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "b6c57219-009b-44a0-bdf8-0d9e51202086", "content-length": "179", "date": "Wed, 30 Apr 2025 06:43:47 GMT"} })
14852025-04-30T06:43:47.896ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1/component/rot/cmpa
14862025-04-30T06:43:47.896ZTRACtest_utilization (service-firewall-rule-progator): fetched allowlist from DB allowed = AllowList { id: 001de000-a110-4000-8000-000000000000, time_created: 2025-04-30T06:06:08.248500Z, time_modified: 2025-04-30T06:06:08.248500Z, allowed_ips: None } background_task = service_firewall_rule_propagation
14872025-04-30T06:43:47.896ZDEBGtest_utilization (service-firewall-rule-progator): Allowlist for user-facing services is set to allow any inbound traffic. Existing VPC firewall rules will not be modified. background_task = service_firewall_rule_propagation
14882025-04-30T06:43:47.896ZDEBGtest_utilization (service-firewall-rule-progator): resolved firewall rules for sled agents background_task = service_firewall_rule_propagation sled_agent_rules = []
14892025-04-30T06:43:47.896ZDEBGtest_utilization (service-firewall-rule-progator): resolved 0 rules for sleds background_task = service_firewall_rule_propagation
14902025-04-30T06:43:47.901ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "e29835fe-6aa1-42cf-9716-c0f438d66ccd", "content-length": "702", "date": "Wed, 30 Apr 2025 06:43:47 GMT"} })
14912025-04-30T06:43:47.901ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1/component/rot/cfpa
14922025-04-30T06:43:47.904ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "acd92532-13bf-4ff8-a18b-77412b50e74f", "content-length": "727", "date": "Wed, 30 Apr 2025 06:43:47 GMT"} })
14932025-04-30T06:43:47.904ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1/component/rot/cfpa
14942025-04-30T06:43:47.906ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "110b51e0-5404-4e98-8d99-304428781841", "content-length": "729", "date": "Wed, 30 Apr 2025 06:43:47 GMT"} })
14952025-04-30T06:43:47.912ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1/component/rot/cfpa
14962025-04-30T06:43:47.915ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "3d8fa7e6-a9e0-4c15-b78d-14c22949436b", "content-length": "728", "date": "Wed, 30 Apr 2025 06:43:47 GMT"} })
14972025-04-30T06:43:47.915ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0
14982025-04-30T06:43:47.917ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "d1bae444-c28e-4c0f-a05b-a28da6f61643", "content-length": "723", "date": "Wed, 30 Apr 2025 06:43:47 GMT"} })
14992025-04-30T06:43:47.950ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0/component/sp/caboose?firmware_slot=0
15002025-04-30T06:43:47.959ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "c1165c1c-31b0-4204-bfca-0028e216f2b1", "content-length": "109", "date": "Wed, 30 Apr 2025 06:43:47 GMT"} })
15012025-04-30T06:43:47.959ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0/component/sp/caboose?firmware_slot=1
15022025-04-30T06:43:47.961ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "123229e5-bcf5-4cc6-a6a3-be8e280f4ac6", "content-length": "109", "date": "Wed, 30 Apr 2025 06:43:47 GMT"} })
15032025-04-30T06:43:47.961ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0/component/rot/caboose?firmware_slot=0
15042025-04-30T06:43:47.970ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "6ad9d64d-1b21-4234-b32f-a0a643a10191", "content-length": "169", "date": "Wed, 30 Apr 2025 06:43:47 GMT"} })
15052025-04-30T06:43:47.971ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0/component/rot/caboose?firmware_slot=1
15062025-04-30T06:43:47.973ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "aeea50cb-c628-4467-b2e4-b155dbee98c0", "content-length": "169", "date": "Wed, 30 Apr 2025 06:43:47 GMT"} })
15072025-04-30T06:43:47.973ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0/component/stage0/caboose?firmware_slot=0
15082025-04-30T06:43:47.975ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "11e06a3f-2fd2-4b9c-a12d-e2836c06dd73", "content-length": "178", "date": "Wed, 30 Apr 2025 06:43:47 GMT"} })
15092025-04-30T06:43:47.976ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0/component/stage0/caboose?firmware_slot=1
15102025-04-30T06:43:47.990ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "d540163f-cbaf-4f24-b971-609044905b90", "content-length": "178", "date": "Wed, 30 Apr 2025 06:43:47 GMT"} })
15112025-04-30T06:43:47.990ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0/component/rot/cmpa
15122025-04-30T06:43:47.991ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "9359c2cd-4a38-4fc1-9225-151925eb6998", "content-length": "702", "date": "Wed, 30 Apr 2025 06:43:47 GMT"} })
15132025-04-30T06:43:47.991ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0/component/rot/cfpa
15142025-04-30T06:43:47.993ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "04e56c31-7f65-41fe-b2fe-4ad3937855ff", "content-length": "727", "date": "Wed, 30 Apr 2025 06:43:47 GMT"} })
15152025-04-30T06:43:47.993ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0/component/rot/cfpa
15162025-04-30T06:43:47.994ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "e0e5ab72-7870-4936-9bd7-151df583d530", "content-length": "729", "date": "Wed, 30 Apr 2025 06:43:47 GMT"} })
15172025-04-30T06:43:47.994ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0/component/rot/cfpa
15182025-04-30T06:43:47.995ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "a5b9d7b5-216b-47db-902e-ad81706cf8b3", "content-length": "728", "date": "Wed, 30 Apr 2025 06:43:47 GMT"} })
15192025-04-30T06:43:47.995ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1
15202025-04-30T06:43:47.997ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1", status: 200, headers: {"content-type": "application/json", "x-request-id": "27c5dbc7-9437-446d-a668-0c43f307ffa9", "content-length": "723", "date": "Wed, 30 Apr 2025 06:43:47 GMT"} })
15212025-04-30T06:43:47.997ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1/component/sp/caboose?firmware_slot=0
15222025-04-30T06:43:48.001ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "dac6dad8-7e8d-499a-995e-1138ad3445ab", "content-length": "109", "date": "Wed, 30 Apr 2025 06:43:47 GMT"} })
15232025-04-30T06:43:48.001ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1/component/sp/caboose?firmware_slot=1
15242025-04-30T06:43:48.021ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "af5623a2-4a5e-49d4-8930-b159adac1e3d", "content-length": "109", "date": "Wed, 30 Apr 2025 06:43:47 GMT"} })
15252025-04-30T06:43:48.021ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1/component/rot/caboose?firmware_slot=0
15262025-04-30T06:43:48.024ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "ba93c92c-cb40-4fd0-8e88-9467e644b1bb", "content-length": "169", "date": "Wed, 30 Apr 2025 06:43:47 GMT"} })
15272025-04-30T06:43:48.031ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1/component/rot/caboose?firmware_slot=1
15282025-04-30T06:43:48.035ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "d20d16fa-5c77-45e1-854c-692d34b5b6c9", "content-length": "169", "date": "Wed, 30 Apr 2025 06:43:47 GMT"} })
15292025-04-30T06:43:48.035ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1/component/stage0/caboose?firmware_slot=0
15302025-04-30T06:43:48.039ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "5d69066b-8e7b-4985-9ed9-1750154ce779", "content-length": "178", "date": "Wed, 30 Apr 2025 06:43:47 GMT"} })
15312025-04-30T06:43:48.039ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1/component/stage0/caboose?firmware_slot=1
15322025-04-30T06:43:48.048ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "86d8079d-54cf-4c06-a57f-9fcdea9ac753", "content-length": "178", "date": "Wed, 30 Apr 2025 06:43:47 GMT"} })
15332025-04-30T06:43:48.048ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1/component/rot/cmpa
15342025-04-30T06:43:48.050ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "de6baa36-88f7-4cc3-8395-5feeaf247ab3", "content-length": "702", "date": "Wed, 30 Apr 2025 06:43:47 GMT"} })
15352025-04-30T06:43:48.050ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1/component/rot/cfpa
15362025-04-30T06:43:48.051ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "936c5ed4-2dd5-4590-893b-bd4f2ed67a87", "content-length": "727", "date": "Wed, 30 Apr 2025 06:43:47 GMT"} })
15372025-04-30T06:43:48.051ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1/component/rot/cfpa
15382025-04-30T06:43:48.052ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "21a9bca2-2055-43ba-b657-8ecc26131d9f", "content-length": "729", "date": "Wed, 30 Apr 2025 06:43:47 GMT"} })
15392025-04-30T06:43:48.052ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1/component/rot/cfpa
15402025-04-30T06:43:48.053ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "d5f9e010-ac99-4e67-bf09-bdedd948e79f", "content-length": "728", "date": "Wed, 30 Apr 2025 06:43:47 GMT"} })
15412025-04-30T06:43:48.053ZTRACtest_utilization (BackgroundTasks): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet
15422025-04-30T06:43:48.053ZTRACtest_utilization (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
15432025-04-30T06:43:48.053ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
15442025-04-30T06:43:48.053ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
15452025-04-30T06:43:48.055ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
15462025-04-30T06:43:48.058ZDEBGtest_utilization (service-firewall-rule-progator): resolved sleds for vpc oxide-services background_task = service_firewall_rule_propagation vpc_to_sled = []
15472025-04-30T06:43:48.058ZDEBGtest_utilization (service-firewall-rule-progator): sending firewall rules to sled agents background_task = service_firewall_rule_propagation
15482025-04-30T06:43:48.058ZINFOtest_utilization (service-firewall-rule-progator): updated firewall rules on 0 sleds background_task = service_firewall_rule_propagation
15492025-04-30T06:43:48.058ZDEBGtest_utilization (service-firewall-rule-progator): successfully propagated service firewall rules background_task = service_firewall_rule_propagation elapsed = 5.816579455s
15502025-04-30T06:43:48.058ZDEBGtest_utilization (BackgroundTasks): activation complete background_task = service_firewall_rule_propagation elapsed = 5.816626569s iteration = 1 status = {}
15512025-04-30T06:43:48.060ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
15522025-04-30T06:43:48.065ZDEBGtest_utilization (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet result = Ok(())
15532025-04-30T06:43:48.065ZTRACtest_utilization (BackgroundTasks): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet
15542025-04-30T06:43:48.065ZTRACtest_utilization (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
15552025-04-30T06:43:48.065ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
15562025-04-30T06:43:48.065ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
15572025-04-30T06:43:48.065ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
15582025-04-30T06:43:48.072ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
15592025-04-30T06:43:48.079ZDEBGtest_utilization (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet result = Ok(())
15602025-04-30T06:43:48.079ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
15612025-04-30T06:43:48.079ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
15622025-04-30T06:43:48.080ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
15632025-04-30T06:43:48.163ZDEBGtest_utilization (BackgroundTasks): begin collection from Sled Agent background_task = inventory_collection sled_agent_url = http://[::1]:39365
15642025-04-30T06:43:48.163ZDEBGtest_utilization (BackgroundTasks): client request SledAgent = http://[::1]:39365 background_task = inventory_collection body = None method = GET uri = http://[::1]:39365/inventory
15652025-04-30T06:43:48.165ZINFOtest_utilization (dropshot): accepted connection local_addr = [::1]:39365 remote_addr = [::1]:41651 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
15662025-04-30T06:43:48.166ZTRACtest_utilization (dropshot): incoming request local_addr = [::1]:39365 method = GET remote_addr = [::1]:41651 req_id = 53ffde32-5e36-4d6f-b155-e2ad315e84e4 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /inventory
15672025-04-30T06:43:48.166ZINFOtest_utilization (dropshot): request completed latency_us = 826 local_addr = [::1]:39365 method = GET remote_addr = [::1]:41651 req_id = 53ffde32-5e36-4d6f-b155-e2ad315e84e4 response_code = 200 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /inventory
15682025-04-30T06:43:48.167ZDEBGtest_utilization (BackgroundTasks): client response SledAgent = http://[::1]:39365 background_task = inventory_collection result = Ok(Response { url: "http://[::1]:39365/inventory", status: 200, headers: {"content-type": "application/json", "x-request-id": "53ffde32-5e36-4d6f-b155-e2ad315e84e4", "content-length": "2769", "date": "Wed, 30 Apr 2025 06:43:47 GMT"} })
15692025-04-30T06:43:48.172ZDEBGtest_utilization (BackgroundTasks): begin collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
15702025-04-30T06:43:48.173ZDEBGtest_utilization (BackgroundTasks): end collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
15712025-04-30T06:43:48.173ZDEBGtest_utilization (BackgroundTasks): finished collection background_task = inventory_collection
15722025-04-30T06:43:48.175ZTRACtest_utilization (BackgroundTasks): authorize begin action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory
15732025-04-30T06:43:48.177ZTRACtest_utilization (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
15742025-04-30T06:43:48.177ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
15752025-04-30T06:43:48.177ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
15762025-04-30T06:43:48.178ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
15772025-04-30T06:43:48.182ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
15782025-04-30T06:43:48.186ZDEBGtest_utilization (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory result = Ok(())
15792025-04-30T06:43:48.186ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
15802025-04-30T06:43:48.186ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
15812025-04-30T06:43:48.187ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
15822025-04-30T06:43:48.188ZINFOtest_utilization (InternalApi): inserted blueprint blueprint_id = 7a85e675-ba78-41e4-9dc6-68e2d561705c
15832025-04-30T06:43:48.188ZTRACtest_utilization (InternalApi): authorize begin action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = BlueprintConfig
15842025-04-30T06:43:48.188ZTRACtest_utilization (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
15852025-04-30T06:43:48.188ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
15862025-04-30T06:43:48.188ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
15872025-04-30T06:43:48.189ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
15882025-04-30T06:43:48.198ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
15892025-04-30T06:43:48.204ZDEBGtest_utilization (InternalApi): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = BlueprintConfig result = Ok(())
15902025-04-30T06:43:48.209ZINFOtest_utilization (InternalApi): No networking records needed for internal_dns service
15912025-04-30T06:43:48.327ZINFOtest_utilization (InternalApi): Inserted networking records for external_dns service
15922025-04-30T06:43:48.327ZINFOtest_utilization (InternalApi): No networking records needed for crucible_pantry service
15932025-04-30T06:43:48.327ZINFOtest_utilization (InternalApi): No networking records needed for cockroach_db service
15942025-04-30T06:43:48.362ZINFOtest_utilization (InternalApi): Inserted networking records for nexus service
15952025-04-30T06:43:48.362ZINFOtest_utilization (InternalApi): No networking records needed for clickhouse service
15962025-04-30T06:43:48.362ZINFOtest_utilization (InternalApi): Inserted service networking records
15972025-04-30T06:43:48.362ZINFOtest_utilization (InternalApi): Inserted physical disks
15982025-04-30T06:43:48.362ZINFOtest_utilization (InternalApi): Inserted zpools
15992025-04-30T06:43:48.362ZINFOtest_utilization (InternalApi): Inserted datasets
16002025-04-30T06:43:48.377ZINFOtest_utilization (InternalApi): Populated DNS tables for internal DNS
16012025-04-30T06:43:48.385ZINFOtest_utilization (InternalApi): Populated DNS tables for external DNS
16022025-04-30T06:43:48.386ZTRACtest_utilization (InternalApi): authorize begin action = CreateChild actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Fleet
16032025-04-30T06:43:48.386ZTRACtest_utilization (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
16042025-04-30T06:43:48.386ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
16052025-04-30T06:43:48.386ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
16062025-04-30T06:43:48.388ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
16072025-04-30T06:43:48.390ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
16082025-04-30T06:43:48.394ZDEBGtest_utilization (InternalApi): authorize result action = CreateChild actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Fleet result = Ok(())
16092025-04-30T06:43:48.394ZTRACtest_utilization (InternalApi): authorize begin action = ModifyPolicy actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Fleet
16102025-04-30T06:43:48.395ZTRACtest_utilization (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
16112025-04-30T06:43:48.395ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
16122025-04-30T06:43:48.395ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
16132025-04-30T06:43:48.396ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
16142025-04-30T06:43:48.398ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
16152025-04-30T06:43:48.401ZDEBGtest_utilization (InternalApi): authorize result action = ModifyPolicy actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Fleet result = Ok(())
16162025-04-30T06:43:48.413ZTRACtest_utilization (InternalApi): authorize begin action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = DnsConfig
16172025-04-30T06:43:48.413ZTRACtest_utilization (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
16182025-04-30T06:43:48.413ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
16192025-04-30T06:43:48.413ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
16202025-04-30T06:43:48.414ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
16212025-04-30T06:43:48.416ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
16222025-04-30T06:43:48.424ZDEBGtest_utilization (InternalApi): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = DnsConfig result = Ok(())
16232025-04-30T06:43:48.424ZTRACtest_utilization (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = DnsConfig
16242025-04-30T06:43:48.424ZTRACtest_utilization (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
16252025-04-30T06:43:48.424ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
16262025-04-30T06:43:48.424ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
16272025-04-30T06:43:48.424ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
16282025-04-30T06:43:48.428ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
16292025-04-30T06:43:48.435ZDEBGtest_utilization (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = DnsConfig result = Ok(())
16302025-04-30T06:43:48.438ZTRACtest_utilization (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = DnsConfig
16312025-04-30T06:43:48.438ZTRACtest_utilization (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
16322025-04-30T06:43:48.438ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
16332025-04-30T06:43:48.438ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
16342025-04-30T06:43:48.440ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
16352025-04-30T06:43:48.453ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
16362025-04-30T06:43:48.470ZDEBGtest_utilization (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = DnsConfig result = Ok(())
16372025-04-30T06:43:48.481ZINFOtest_utilization (InternalApi): Created recovery silo
16382025-04-30T06:43:48.486ZINFOtest_utilization (InternalApi): Created recovery user
16392025-04-30T06:43:48.490ZINFOtest_utilization (InternalApi): Created recovery user's password
16402025-04-30T06:43:48.490ZTRACtest_utilization (InternalApi): authorize begin action = ModifyPolicy actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Silo { parent: Fleet, key: ad91dcf9-5eb8-4a8d-b721-b5a47ceac136, lookup_type: ById(ad91dcf9-5eb8-4a8d-b721-b5a47ceac136) }
16412025-04-30T06:43:48.490ZTRACtest_utilization (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = ad91dcf9-5eb8-4a8d-b721-b5a47ceac136 resource_type = Silo
16422025-04-30T06:43:48.490ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
16432025-04-30T06:43:48.490ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
16442025-04-30T06:43:48.491ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
16452025-04-30T06:43:48.495ZTRACtest_utilization (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
16462025-04-30T06:43:48.495ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
16472025-04-30T06:43:48.495ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
16482025-04-30T06:43:48.495ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
16492025-04-30T06:43:48.497ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
16502025-04-30T06:43:48.506ZDEBGtest_utilization (InternalApi): authorize result action = ModifyPolicy actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Silo { parent: Fleet, key: ad91dcf9-5eb8-4a8d-b721-b5a47ceac136, lookup_type: ById(ad91dcf9-5eb8-4a8d-b721-b5a47ceac136) } result = Ok(())
16512025-04-30T06:43:48.506ZDEBGtest_utilization (InternalApi): Generated role assignment queries
16522025-04-30T06:43:48.515ZINFOtest_utilization (InternalApi): Granted Silo privileges
16532025-04-30T06:43:48.515ZTRACtest_utilization (InternalApi): authorize begin action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Fleet
16542025-04-30T06:43:48.515ZTRACtest_utilization (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
16552025-04-30T06:43:48.515ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
16562025-04-30T06:43:48.515ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
16572025-04-30T06:43:48.516ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
16582025-04-30T06:43:48.519ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
16592025-04-30T06:43:48.522ZDEBGtest_utilization (InternalApi): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Fleet result = Ok(())
16602025-04-30T06:43:48.537ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
16612025-04-30T06:43:48.537ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
16622025-04-30T06:43:48.537ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
16632025-04-30T06:43:48.554ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
16642025-04-30T06:43:48.554ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
16652025-04-30T06:43:48.556ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
16662025-04-30T06:43:48.558ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
16672025-04-30T06:43:48.558ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
16682025-04-30T06:43:48.559ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
16692025-04-30T06:43:48.562ZTRACtest_utilization (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) }
16702025-04-30T06:43:48.562ZTRACtest_utilization (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-4401-4000-8000-000000000000 resource_type = Project
16712025-04-30T06:43:48.562ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
16722025-04-30T06:43:48.562ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
16732025-04-30T06:43:48.563ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
16742025-04-30T06:43:48.575ZTRACtest_utilization (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-5110-4000-8000-000000000001 resource_type = Silo
16752025-04-30T06:43:48.575ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
16762025-04-30T06:43:48.575ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
16772025-04-30T06:43:48.575ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
16782025-04-30T06:43:48.583ZTRACtest_utilization (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
16792025-04-30T06:43:48.585ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
16802025-04-30T06:43:48.585ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
16812025-04-30T06:43:48.585ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
16822025-04-30T06:43:48.598ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
16832025-04-30T06:43:48.615ZDEBGtest_utilization (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
16842025-04-30T06:43:48.615ZTRACtest_utilization (InternalApi): successful lookup of siloed resource "Vpc" using built-in user
16852025-04-30T06:43:48.615ZTRACtest_utilization (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) }
16862025-04-30T06:43:48.615ZTRACtest_utilization (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-4401-4000-8000-000000000000 resource_type = Project
16872025-04-30T06:43:48.615ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
16882025-04-30T06:43:48.615ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
16892025-04-30T06:43:48.616ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
16902025-04-30T06:43:48.622ZTRACtest_utilization (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-5110-4000-8000-000000000001 resource_type = Silo
16912025-04-30T06:43:48.622ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
16922025-04-30T06:43:48.622ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
16932025-04-30T06:43:48.623ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
16942025-04-30T06:43:48.640ZTRACtest_utilization (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
16952025-04-30T06:43:48.640ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
16962025-04-30T06:43:48.640ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
16972025-04-30T06:43:48.641ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
16982025-04-30T06:43:48.652ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
16992025-04-30T06:43:48.670ZDEBGtest_utilization (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
17002025-04-30T06:43:48.670ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
17012025-04-30T06:43:48.672ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
17022025-04-30T06:43:48.672ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
17032025-04-30T06:43:48.685ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
17042025-04-30T06:43:48.685ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
17052025-04-30T06:43:48.686ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
17062025-04-30T06:43:48.691ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
17072025-04-30T06:43:48.691ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
17082025-04-30T06:43:48.693ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
17092025-04-30T06:43:48.699ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
17102025-04-30T06:43:48.699ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
17112025-04-30T06:43:48.700ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
17122025-04-30T06:43:48.704ZTRACtest_utilization (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) }
17132025-04-30T06:43:48.704ZTRACtest_utilization (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-4401-4000-8000-000000000000 resource_type = Project
17142025-04-30T06:43:48.704ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
17152025-04-30T06:43:48.704ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
17162025-04-30T06:43:48.705ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
17172025-04-30T06:43:48.706ZINFOtest_utilization (BackgroundTasks): inserted inventory collection background_task = inventory_collection collection_id = 13d8b8d9-6c6b-47a6-9190-952544edb75b
17182025-04-30T06:43:48.706ZDEBGtest_utilization (BackgroundTasks): inventory collection complete background_task = inventory_collection collection_id = 13d8b8d9-6c6b-47a6-9190-952544edb75b time_started = 2025-04-30 06:43:47.112108 UTC
17192025-04-30T06:43:48.706ZDEBGtest_utilization (BackgroundTasks): activation complete background_task = inventory_collection elapsed = 5.403157769s iteration = 2 status = {"collection_id":"13d8b8d9-6c6b-47a6-9190-952544edb75b","time_done":"2025-04-30 06:43:48.173640 UTC","time_started":"2025-04-30 06:43:47.112108 UTC"}
17202025-04-30T06:43:48.706ZDEBGtest_utilization (BackgroundTasks): activating background_task = blueprint_rendezvous iteration = 2 reason = Dependency
17212025-04-30T06:43:48.706ZWARNtest_utilization (BackgroundTasks): Blueprint rendezvous: skipped background_task = blueprint_rendezvous reason = no blueprint
17222025-04-30T06:43:48.706ZDEBGtest_utilization (BackgroundTasks): activation complete background_task = blueprint_rendezvous elapsed = 27.454\u{b5}s iteration = 2 status = {"error":"no blueprint"}
17232025-04-30T06:43:48.706ZDEBGtest_utilization (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 2 reason = Dependency
17242025-04-30T06:43:48.706ZDEBGtest_utilization (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 10.662\u{b5}s iteration = 2 status = {"error":"task disabled"}
17252025-04-30T06:43:48.708ZTRACtest_utilization (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-5110-4000-8000-000000000001 resource_type = Silo
17262025-04-30T06:43:48.708ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
17272025-04-30T06:43:48.708ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
17282025-04-30T06:43:48.709ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
17292025-04-30T06:43:48.712ZTRACtest_utilization (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
17302025-04-30T06:43:48.712ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
17312025-04-30T06:43:48.712ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
17322025-04-30T06:43:48.713ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
17332025-04-30T06:43:48.715ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
17342025-04-30T06:43:48.724ZDEBGtest_utilization (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
17352025-04-30T06:43:48.724ZTRACtest_utilization (InternalApi): successful lookup of siloed resource "Vpc" using built-in user
17362025-04-30T06:43:48.724ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
17372025-04-30T06:43:48.724ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
17382025-04-30T06:43:48.725ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
17392025-04-30T06:43:48.731ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
17402025-04-30T06:43:48.731ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
17412025-04-30T06:43:48.732ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
17422025-04-30T06:43:48.737ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
17432025-04-30T06:43:48.737ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
17442025-04-30T06:43:48.738ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
17452025-04-30T06:43:48.743ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
17462025-04-30T06:43:48.743ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
17472025-04-30T06:43:48.744ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
17482025-04-30T06:43:48.749ZTRACtest_utilization (InternalApi): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000002, lookup_type: ByName("nexus") }
17492025-04-30T06:43:48.750ZTRACtest_utilization (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-4401-4000-8000-000000000000 resource_type = Project
17502025-04-30T06:43:48.750ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
17512025-04-30T06:43:48.750ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
17522025-04-30T06:43:48.751ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
17532025-04-30T06:43:48.754ZTRACtest_utilization (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-5110-4000-8000-000000000001 resource_type = Silo
17542025-04-30T06:43:48.754ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
17552025-04-30T06:43:48.754ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
17562025-04-30T06:43:48.754ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
17572025-04-30T06:43:48.757ZTRACtest_utilization (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
17582025-04-30T06:43:48.757ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
17592025-04-30T06:43:48.757ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
17602025-04-30T06:43:48.758ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
17612025-04-30T06:43:48.761ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
17622025-04-30T06:43:48.769ZDEBGtest_utilization (InternalApi): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000002, lookup_type: ByName("nexus") } result = Ok(())
17632025-04-30T06:43:48.769ZTRACtest_utilization (InternalApi): successful lookup of siloed resource "VpcSubnet" using built-in user
17642025-04-30T06:43:48.769ZTRACtest_utilization (InternalApi): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000002, lookup_type: ByName("nexus") }
17652025-04-30T06:43:48.769ZTRACtest_utilization (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-4401-4000-8000-000000000000 resource_type = Project
17662025-04-30T06:43:48.769ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
17672025-04-30T06:43:48.769ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
17682025-04-30T06:43:48.770ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
17692025-04-30T06:43:48.773ZTRACtest_utilization (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-5110-4000-8000-000000000001 resource_type = Silo
17702025-04-30T06:43:48.773ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
17712025-04-30T06:43:48.773ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
17722025-04-30T06:43:48.773ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
17732025-04-30T06:43:48.776ZTRACtest_utilization (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
17742025-04-30T06:43:48.776ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
17752025-04-30T06:43:48.776ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
17762025-04-30T06:43:48.776ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
17772025-04-30T06:43:48.779ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
17782025-04-30T06:43:48.793ZDEBGtest_utilization (InternalApi): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000002, lookup_type: ByName("nexus") } result = Ok(())
17792025-04-30T06:43:48.794ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
17802025-04-30T06:43:48.794ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
17812025-04-30T06:43:48.794ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
17822025-04-30T06:43:48.800ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
17832025-04-30T06:43:48.800ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
17842025-04-30T06:43:48.801ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
17852025-04-30T06:43:48.803ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
17862025-04-30T06:43:48.804ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
17872025-04-30T06:43:48.804ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
17882025-04-30T06:43:48.807ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
17892025-04-30T06:43:48.807ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
17902025-04-30T06:43:48.809ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
17912025-04-30T06:43:48.812ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
17922025-04-30T06:43:48.812ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
17932025-04-30T06:43:48.813ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
17942025-04-30T06:43:48.818ZTRACtest_utilization (InternalApi): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000001, lookup_type: ByName("external-dns") }
17952025-04-30T06:43:48.818ZTRACtest_utilization (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-4401-4000-8000-000000000000 resource_type = Project
17962025-04-30T06:43:48.818ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
17972025-04-30T06:43:48.818ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
17982025-04-30T06:43:48.823ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
17992025-04-30T06:43:48.826ZTRACtest_utilization (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-5110-4000-8000-000000000001 resource_type = Silo
18002025-04-30T06:43:48.826ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
18012025-04-30T06:43:48.826ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
18022025-04-30T06:43:48.828ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
18032025-04-30T06:43:48.832ZTRACtest_utilization (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
18042025-04-30T06:43:48.832ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
18052025-04-30T06:43:48.832ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
18062025-04-30T06:43:48.832ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
18072025-04-30T06:43:48.836ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
18082025-04-30T06:43:48.848ZDEBGtest_utilization (InternalApi): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000001, lookup_type: ByName("external-dns") } result = Ok(())
18092025-04-30T06:43:48.848ZTRACtest_utilization (InternalApi): successful lookup of siloed resource "VpcSubnet" using built-in user
18102025-04-30T06:43:48.848ZTRACtest_utilization (InternalApi): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000001, lookup_type: ByName("external-dns") }
18112025-04-30T06:43:48.848ZTRACtest_utilization (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-4401-4000-8000-000000000000 resource_type = Project
18122025-04-30T06:43:48.848ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
18132025-04-30T06:43:48.848ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
18142025-04-30T06:43:48.849ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
18152025-04-30T06:43:48.854ZTRACtest_utilization (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-5110-4000-8000-000000000001 resource_type = Silo
18162025-04-30T06:43:48.854ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
18172025-04-30T06:43:48.854ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
18182025-04-30T06:43:48.854ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
18192025-04-30T06:43:48.857ZTRACtest_utilization (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
18202025-04-30T06:43:48.857ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
18212025-04-30T06:43:48.857ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
18222025-04-30T06:43:48.859ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
18232025-04-30T06:43:48.863ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
18242025-04-30T06:43:48.878ZDEBGtest_utilization (InternalApi): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000001, lookup_type: ByName("external-dns") } result = Ok(())
18252025-04-30T06:43:48.878ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
18262025-04-30T06:43:48.878ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
18272025-04-30T06:43:48.878ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
18282025-04-30T06:43:48.889ZDEBGtest_utilization (nexus): resolved names for firewall rules instance_interfaces = {} subnet_interfaces = {Name("nexus"): [NetworkInterface { id: 46961afd-00ab-4b86-9fd5-30529489e593, kind: Service { id: ac55d8d9-a298-4534-898b-beee1c9e4e52 }, name: Name("nexus-ac55d8d9-a298-4534-898b-beee1c9e4e52"), ip: 172.30.2.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 128, 1])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], Name("external-dns"): [NetworkInterface { id: e9a61db4-ad4d-462b-b2bd-03be871e2ce1, kind: Service { id: 47c428f0-ed10-4268-b847-550bb644ba22 }, name: Name("external-dns-47c428f0-ed10-4268-b847-550bb644ba22"), ip: 172.30.1.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 128, 0])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }]} subnet_networks = {Name("nexus"): [V4(Ipv4Network { addr: 172.30.2.0, prefix: 24 }), V6(Ipv6Network { addr: fd77:e9d2:9cd9:2::, prefix: 64 })], Name("external-dns"): [V4(Ipv4Network { addr: 172.30.1.0, prefix: 24 }), V6(Ipv6Network { addr: fd77:e9d2:9cd9:1::, prefix: 64 })]} vpc_interfaces = {}
18292025-04-30T06:43:48.889ZTRACtest_utilization (InternalApi): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Fleet
18302025-04-30T06:43:48.889ZTRACtest_utilization (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
18312025-04-30T06:43:48.889ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
18322025-04-30T06:43:48.889ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
18332025-04-30T06:43:48.890ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
18342025-04-30T06:43:48.893ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
18352025-04-30T06:43:48.900ZDEBGtest_utilization (InternalApi): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Fleet result = Ok(())
18362025-04-30T06:43:48.900ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
18372025-04-30T06:43:48.900ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
18382025-04-30T06:43:48.900ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
18392025-04-30T06:43:48.904ZTRACtest_utilization (nexus): fetched allowlist from DB allowed = AllowList { id: 001de000-a110-4000-8000-000000000000, time_created: 2025-04-30T06:06:08.248500Z, time_modified: 2025-04-30T06:43:48.522049Z, allowed_ips: None }
18402025-04-30T06:43:48.904ZDEBGtest_utilization (nexus): Allowlist for user-facing services is set to allow any inbound traffic. Existing VPC firewall rules will not be modified.
18412025-04-30T06:43:48.906ZDEBGtest_utilization (nexus): resolved firewall rules for sled agents sled_agent_rules = [ResolvedVpcFirewallRule { action: Allow, direction: Inbound, filter_hosts: None, filter_ports: Some([L4PortRange("53")]), filter_protocols: Some([Udp]), priority: 65534, status: Enabled, targets: [NetworkInterface { id: e9a61db4-ad4d-462b-b2bd-03be871e2ce1, kind: Service { id: 47c428f0-ed10-4268-b847-550bb644ba22 }, name: Name("external-dns-47c428f0-ed10-4268-b847-550bb644ba22"), ip: 172.30.1.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 128, 0])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }] }, ResolvedVpcFirewallRule { action: Allow, direction: Inbound, filter_hosts: None, filter_ports: Some([L4PortRange("80"), L4PortRange("443")]), filter_protocols: Some([Tcp]), priority: 65534, status: Enabled, targets: [NetworkInterface { id: 46961afd-00ab-4b86-9fd5-30529489e593, kind: Service { id: ac55d8d9-a298-4534-898b-beee1c9e4e52 }, name: Name("nexus-ac55d8d9-a298-4534-898b-beee1c9e4e52"), ip: 172.30.2.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 128, 1])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }] }]
18422025-04-30T06:43:48.906ZDEBGtest_utilization (nexus): resolved 2 rules for sleds
18432025-04-30T06:43:49.039ZDEBGtest_utilization (nexus): resolved sleds for vpc oxide-services vpc_to_sled = [Sled { identity: SledIdentity { id: b6d65341-167c-41df-9b5c-41cded99c229, time_created: 2025-04-30T06:43:42.325143Z, time_modified: 2025-04-30T06:43:42.325143Z }, time_deleted: None, rcgen: Generation(Generation(1)), rack_id: c19a698f-c6f9-4a17-ae30-20d711b8f7dc, is_scrimlet: true, serial_number: "sim-b6d65341", part_number: "Unknown", revision: SqlU32(0), usable_hardware_threads: SqlU32(16), usable_physical_ram: ByteCount(ByteCount(34359738368)), reservoir_size: ByteCount(ByteCount(17179869184)), ip: ::1, port: SqlU16(39365), last_used_address: ::ffff, policy: InService, state: Active, sled_agent_gen: Generation(Generation(1)), repo_depot_port: SqlU16(51171) }]
18442025-04-30T06:43:49.046ZDEBGtest_utilization (nexus): sending firewall rules to sled agents
18452025-04-30T06:43:49.047ZTRACtest_utilization (InstanceAllocator): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000004ead, .. }) resource = Database
18462025-04-30T06:43:49.047ZDEBGtest_utilization (InstanceAllocator): roles roles = RoleSet { roles: {} }
18472025-04-30T06:43:49.048ZDEBGtest_utilization (InstanceAllocator): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000004ead, .. }) resource = Database result = Ok(())
18482025-04-30T06:43:49.053ZTRACtest_utilization (InstanceAllocator): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000004ead, .. }) resource = Sled { parent: Fleet, key: b6d65341-167c-41df-9b5c-41cded99c229, lookup_type: ById(b6d65341-167c-41df-9b5c-41cded99c229) }
18492025-04-30T06:43:49.053ZTRACtest_utilization (InstanceAllocator): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000004ead, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
18502025-04-30T06:43:49.054ZTRACtest_utilization (InstanceAllocator): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000004ead, .. }) resource = Database
18512025-04-30T06:43:49.054ZDEBGtest_utilization (InstanceAllocator): roles roles = RoleSet { roles: {} }
18522025-04-30T06:43:49.055ZDEBGtest_utilization (InstanceAllocator): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000004ead, .. }) resource = Database result = Ok(())
18532025-04-30T06:43:49.060ZDEBGtest_utilization (InstanceAllocator): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "viewer")} }
18542025-04-30T06:43:49.064ZDEBGtest_utilization (InstanceAllocator): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000004ead, .. }) resource = Sled { parent: Fleet, key: b6d65341-167c-41df-9b5c-41cded99c229, lookup_type: ById(b6d65341-167c-41df-9b5c-41cded99c229) } result = Ok(())
18552025-04-30T06:43:49.119ZDEBGtest_utilization (nexus): client request SledAgent = b6d65341-167c-41df-9b5c-41cded99c229 body = Some(Body) method = PUT uri = http://[::1]:39365/vpc/001de000-074c-4000-8000-000000000000/firewall/rules
18562025-04-30T06:43:49.120ZINFOtest_utilization (dropshot): accepted connection local_addr = [::1]:39365 remote_addr = [::1]:58163 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
18572025-04-30T06:43:49.121ZTRACtest_utilization (dropshot): incoming request local_addr = [::1]:39365 method = PUT remote_addr = [::1]:58163 req_id = 53dd488f-4a41-4c9d-953b-898c70566b4a sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /vpc/001de000-074c-4000-8000-000000000000/firewall/rules
18582025-04-30T06:43:49.122ZINFOtest_utilization (dropshot): request completed latency_us = 1352 local_addr = [::1]:39365 method = PUT remote_addr = [::1]:58163 req_id = 53dd488f-4a41-4c9d-953b-898c70566b4a response_code = 204 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /vpc/001de000-074c-4000-8000-000000000000/firewall/rules
18592025-04-30T06:43:49.122ZDEBGtest_utilization (nexus): client response SledAgent = b6d65341-167c-41df-9b5c-41cded99c229 result = Ok(Response { url: "http://[::1]:39365/vpc/001de000-074c-4000-8000-000000000000/firewall/rules", status: 204, headers: {"x-request-id": "53dd488f-4a41-4c9d-953b-898c70566b4a", "date": "Wed, 30 Apr 2025 06:43:49 GMT"} })
18602025-04-30T06:43:49.125ZINFOtest_utilization (nexus): updated firewall rules on 1 sleds
18612025-04-30T06:43:49.125ZTRACtest_utilization (ServiceBalancer): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-00000000bac3, .. }) resource = Database
18622025-04-30T06:43:49.125ZDEBGtest_utilization (ServiceBalancer): roles roles = RoleSet { roles: {} }
18632025-04-30T06:43:49.126ZDEBGtest_utilization (ServiceBalancer): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-00000000bac3, .. }) resource = Database result = Ok(())
18642025-04-30T06:43:49.126ZDEBGtest_utilization (BackgroundTasks): activating background_task = dns_config_internal dns_group = internal iteration = 2 reason = Signaled
18652025-04-30T06:43:49.126ZDEBGtest_utilization (BackgroundTasks): reading DNS version background_task = dns_config_internal dns_group = internal
18662025-04-30T06:43:49.126ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database
18672025-04-30T06:43:49.126ZDEBGtest_utilization (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
18682025-04-30T06:43:49.128ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
18692025-04-30T06:43:49.128ZDEBGtest_utilization (BackgroundTasks): activating background_task = dns_servers_internal dns_group = internal iteration = 2 reason = Signaled
18702025-04-30T06:43:49.128ZTRACtest_utilization (DnsResolver): lookup_all_socket_v6 srv dns_name = _nameservice._tcp.control-plane.oxide.internal
18712025-04-30T06:43:49.128ZDEBGtest_utilization (BackgroundTasks): activating background_task = dns_config_external dns_group = external iteration = 2 reason = Signaled
18722025-04-30T06:43:49.128ZDEBGtest_utilization (BackgroundTasks): reading DNS version background_task = dns_config_external dns_group = external
18732025-04-30T06:43:49.128ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database
18742025-04-30T06:43:49.128ZDEBGtest_utilization (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
18752025-04-30T06:43:49.129ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
18762025-04-30T06:43:49.129ZDEBGtest_utilization (BackgroundTasks): activating background_task = dns_servers_external dns_group = external iteration = 2 reason = Signaled
18772025-04-30T06:43:49.129ZTRACtest_utilization (DnsResolver): lookup_all_socket_v6 srv dns_name = _external-dns._tcp.control-plane.oxide.internal
18782025-04-30T06:43:49.129ZDEBGtest_utilization (BackgroundTasks): activating background_task = external_endpoints iteration = 2 reason = Signaled
18792025-04-30T06:43:49.129ZTRACtest_utilization (BackgroundTasks): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet
18802025-04-30T06:43:49.130ZTRACtest_utilization (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = external_endpoints resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
18812025-04-30T06:43:49.130ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database
18822025-04-30T06:43:49.130ZDEBGtest_utilization (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
18832025-04-30T06:43:49.130ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
18842025-04-30T06:43:49.130ZDEBGtest_utilization (BackgroundTasks): activating background_task = inventory_collection iteration = 3 reason = Signaled
18852025-04-30T06:43:49.131ZTRACtest_utilization (BackgroundTasks): authorize begin action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory
18862025-04-30T06:43:49.131ZTRACtest_utilization (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
18872025-04-30T06:43:49.131ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
18882025-04-30T06:43:49.131ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
18892025-04-30T06:43:49.133ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
18902025-04-30T06:43:49.134ZTRACtest_utilization (dns): buffer buffer = Length: 75 (0x4b) bytes\n0000: 90 78 01 00 00 01 00 00 00 00 00 01 0c 5f 6e 61 .x..........._na\n0010: 6d 65 73 65 72 76 69 63 65 04 5f 74 63 70 0d 63 meservice._tcp.c\n0020: 6f 6e 74 72 6f 6c 2d 70 6c 61 6e 65 05 6f 78 69 ontrol-plane.oxi\n0030: 64 65 08 69 6e 74 65 72 6e 61 6c 00 00 21 00 01 de.internal..!..\n0040: 00 00 29 04 d0 00 00 00 00 00 00 ..)........ kind = dns peer_addr = [::1]:63563 req_id = f5938744-d7fa-4195-9cfd-654b7c3cb66d
18912025-04-30T06:43:49.134ZDEBGtest_utilization (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 36984,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_nameservice._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_nameservice._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 12,\n 95,\n 110,\n 97,\n 109,\n 101,\n 115,\n 101,\n 114,\n 118,\n 105,\n 99,\n 101,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:63563 req_id = f5938744-d7fa-4195-9cfd-654b7c3cb66d
18922025-04-30T06:43:49.135ZDEBGtest_utilization (store): query key key = _nameservice._tcp
18932025-04-30T06:43:49.135ZDEBGtest_utilization (store): query key key = 2ec6b9d0-21c7-4ff3-a311-7a18b348aa25.host
18942025-04-30T06:43:49.135ZDEBGtest_utilization (dns): dns response additional_records = [Record { name_labels: Name("2ec6b9d0-21c7-4ff3-a311-7a18b348aa25.host.control-plane.oxide.internal"), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] kind = dns peer_addr = [::1]:63563 query = LowerQuery { name: LowerName(Name("_nameservice._tcp.control-plane.oxide.internal.")), original: Query { name: Name("_nameservice._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN } } records = [Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 42391, target: Name("2ec6b9d0-21c7-4ff3-a311-7a18b348aa25.host.control-plane.oxide.internal") })) }] req_id = f5938744-d7fa-4195-9cfd-654b7c3cb66d
18952025-04-30T06:43:49.135ZTRACtest_utilization (dns): buffer buffer = Length: 76 (0x4c) bytes\n0000: 44 65 01 00 00 01 00 00 00 00 00 01 0d 5f 65 78 De..........._ex\n0010: 74 65 72 6e 61 6c 2d 64 6e 73 04 5f 74 63 70 0d ternal-dns._tcp.\n0020: 63 6f 6e 74 72 6f 6c 2d 70 6c 61 6e 65 05 6f 78 control-plane.ox\n0030: 69 64 65 08 69 6e 74 65 72 6e 61 6c 00 00 21 00 ide.internal..!.\n0040: 01 00 00 29 04 d0 00 00 00 00 00 00 ...)........ kind = dns peer_addr = [::1]:56350 req_id = 4dca5ae5-4aa4-4e3c-8c9d-82b8000e36de
18962025-04-30T06:43:49.135ZDEBGtest_utilization (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 17509,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_external-dns._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_external-dns._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 13,\n 95,\n 101,\n 120,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 45,\n 100,\n 110,\n 115,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:56350 req_id = 4dca5ae5-4aa4-4e3c-8c9d-82b8000e36de
18972025-04-30T06:43:49.135ZDEBGtest_utilization (store): query key key = _external-dns._tcp
18982025-04-30T06:43:49.135ZDEBGtest_utilization (store): query key key = 47c428f0-ed10-4268-b847-550bb644ba22.host
18992025-04-30T06:43:49.135ZDEBGtest_utilization (dns): dns response additional_records = [Record { name_labels: Name("47c428f0-ed10-4268-b847-550bb644ba22.host.control-plane.oxide.internal"), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] kind = dns peer_addr = [::1]:56350 query = LowerQuery { name: LowerName(Name("_external-dns._tcp.control-plane.oxide.internal.")), original: Query { name: Name("_external-dns._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN } } records = [Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 48132, target: Name("47c428f0-ed10-4268-b847-550bb644ba22.host.control-plane.oxide.internal") })) }] req_id = 4dca5ae5-4aa4-4e3c-8c9d-82b8000e36de
19002025-04-30T06:43:49.137ZDEBGtest_utilization (DnsResolver): lookup_all_socket_v6 srv dns_name = _nameservice._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_nameservice._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 42391, target: Name("2ec6b9d0-21c7-4ff3-a311-7a18b348aa25.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("2ec6b9d0-21c7-4ff3-a311-7a18b348aa25.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }], valid_until: Instant { tv_sec: 4293, tv_nsec: 936797480 } })
19012025-04-30T06:43:49.137ZTRACtest_utilization (DnsResolver): lookup_service_targets: looking up SRV target
19022025-04-30T06:43:49.138ZDEBGtest_utilization (DnsResolver): lookup_all_socket_v6 srv dns_name = _external-dns._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_external-dns._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 48132, target: Name("47c428f0-ed10-4268-b847-550bb644ba22.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("47c428f0-ed10-4268-b847-550bb644ba22.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }], valid_until: Instant { tv_sec: 4293, tv_nsec: 937183689 } })
19032025-04-30T06:43:49.138ZTRACtest_utilization (DnsResolver): lookup_service_targets: looking up SRV target
19042025-04-30T06:43:49.138ZTRACtest_utilization (BackgroundTasks): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig
19052025-04-30T06:43:49.138ZTRACtest_utilization (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = dns_config_internal dns_group = internal resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
19062025-04-30T06:43:49.138ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database
19072025-04-30T06:43:49.138ZDEBGtest_utilization (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
19082025-04-30T06:43:49.141ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
19092025-04-30T06:43:49.141ZTRACtest_utilization (ServiceBalancer): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-00000000bac3, .. }) resource = Rack { parent: Fleet, key: c19a698f-c6f9-4a17-ae30-20d711b8f7dc, lookup_type: ById(c19a698f-c6f9-4a17-ae30-20d711b8f7dc) }
19102025-04-30T06:43:49.141ZTRACtest_utilization (ServiceBalancer): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-00000000bac3, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
19112025-04-30T06:43:49.141ZTRACtest_utilization (ServiceBalancer): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-00000000bac3, .. }) resource = Database
19122025-04-30T06:43:49.141ZDEBGtest_utilization (ServiceBalancer): roles roles = RoleSet { roles: {} }
19132025-04-30T06:43:49.142ZDEBGtest_utilization (ServiceBalancer): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-00000000bac3, .. }) resource = Database result = Ok(())
19142025-04-30T06:43:49.142ZTRACtest_utilization (BackgroundTasks): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig
19152025-04-30T06:43:49.142ZTRACtest_utilization (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = dns_config_external dns_group = external resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
19162025-04-30T06:43:49.142ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database
19172025-04-30T06:43:49.143ZDEBGtest_utilization (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
19182025-04-30T06:43:49.143ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
19192025-04-30T06:43:49.143ZTRACtest_utilization (dns): buffer buffer = Length: 99 (0x63) bytes\n0000: e2 ca 01 00 00 01 00 00 00 00 00 01 24 32 65 63 ............$2ec\n0010: 36 62 39 64 30 2d 32 31 63 37 2d 34 66 66 33 2d 6b9d0-21c7-4ff3-\n0020: 61 33 31 31 2d 37 61 31 38 62 33 34 38 61 61 32 a311-7a18b348aa2\n0030: 35 04 68 6f 73 74 0d 63 6f 6e 74 72 6f 6c 2d 70 5.host.control-p\n0040: 6c 61 6e 65 05 6f 78 69 64 65 08 69 6e 74 65 72 lane.oxide.inter\n0050: 6e 61 6c 00 00 1c 00 01 00 00 29 04 d0 00 00 00 nal.......).....\n0060: 00 00 00 ... kind = dns peer_addr = [::1]:53635 req_id = c0adfd93-9640-4ba3-ad4c-8d925fdfe52c
19202025-04-30T06:43:49.144ZDEBGtest_utilization (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 58058,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("2ec6b9d0-21c7-4ff3-a311-7a18b348aa25.host.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("2ec6b9d0-21c7-4ff3-a311-7a18b348aa25.host.control-plane.oxide.internal."),\n query_type: AAAA,\n query_class: IN,\n },\n },\n original: [\n 36,\n 50,\n 101,\n 99,\n 54,\n 98,\n 57,\n 100,\n 48,\n 45,\n 50,\n 49,\n 99,\n 55,\n 45,\n 52,\n 102,\n 102,\n 51,\n 45,\n 97,\n 51,\n 49,\n 49,\n 45,\n 55,\n 97,\n 49,\n 56,\n 98,\n 51,\n 52,\n 56,\n 97,\n 97,\n 50,\n 53,\n 4,\n 104,\n 111,\n 115,\n 116,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 28,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:53635 req_id = c0adfd93-9640-4ba3-ad4c-8d925fdfe52c
19212025-04-30T06:43:49.144ZDEBGtest_utilization (store): query key key = 2ec6b9d0-21c7-4ff3-a311-7a18b348aa25.host
19222025-04-30T06:43:49.144ZDEBGtest_utilization (dns): dns response additional_records = [] kind = dns peer_addr = [::1]:53635 query = LowerQuery { name: LowerName(Name("2ec6b9d0-21c7-4ff3-a311-7a18b348aa25.host.control-plane.oxide.internal.")), original: Query { name: Name("2ec6b9d0-21c7-4ff3-a311-7a18b348aa25.host.control-plane.oxide.internal."), query_type: AAAA, query_class: IN } } records = [Record { name_labels: Name("2ec6b9d0-21c7-4ff3-a311-7a18b348aa25.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] req_id = c0adfd93-9640-4ba3-ad4c-8d925fdfe52c
19232025-04-30T06:43:49.144ZTRACtest_utilization (dns): buffer buffer = Length: 99 (0x63) bytes\n0000: fd c4 01 00 00 01 00 00 00 00 00 01 24 34 37 63 ............$47c\n0010: 34 32 38 66 30 2d 65 64 31 30 2d 34 32 36 38 2d 428f0-ed10-4268-\n0020: 62 38 34 37 2d 35 35 30 62 62 36 34 34 62 61 32 b847-550bb644ba2\n0030: 32 04 68 6f 73 74 0d 63 6f 6e 74 72 6f 6c 2d 70 2.host.control-p\n0040: 6c 61 6e 65 05 6f 78 69 64 65 08 69 6e 74 65 72 lane.oxide.inter\n0050: 6e 61 6c 00 00 1c 00 01 00 00 29 04 d0 00 00 00 nal.......).....\n0060: 00 00 00 ... kind = dns peer_addr = [::1]:61695 req_id = ddc3f64f-f47d-4189-8e15-a7b53b3bbb68
19242025-04-30T06:43:49.145ZDEBGtest_utilization (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 64964,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("47c428f0-ed10-4268-b847-550bb644ba22.host.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("47c428f0-ed10-4268-b847-550bb644ba22.host.control-plane.oxide.internal."),\n query_type: AAAA,\n query_class: IN,\n },\n },\n original: [\n 36,\n 52,\n 55,\n 99,\n 52,\n 50,\n 56,\n 102,\n 48,\n 45,\n 101,\n 100,\n 49,\n 48,\n 45,\n 52,\n 50,\n 54,\n 56,\n 45,\n 98,\n 56,\n 52,\n 55,\n 45,\n 53,\n 53,\n 48,\n 98,\n 98,\n 54,\n 52,\n 52,\n 98,\n 97,\n 50,\n 50,\n 4,\n 104,\n 111,\n 115,\n 116,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 28,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:61695 req_id = ddc3f64f-f47d-4189-8e15-a7b53b3bbb68
19252025-04-30T06:43:49.145ZDEBGtest_utilization (store): query key key = 47c428f0-ed10-4268-b847-550bb644ba22.host
19262025-04-30T06:43:49.145ZDEBGtest_utilization (dns): dns response additional_records = [] kind = dns peer_addr = [::1]:61695 query = LowerQuery { name: LowerName(Name("47c428f0-ed10-4268-b847-550bb644ba22.host.control-plane.oxide.internal.")), original: Query { name: Name("47c428f0-ed10-4268-b847-550bb644ba22.host.control-plane.oxide.internal."), query_type: AAAA, query_class: IN } } records = [Record { name_labels: Name("47c428f0-ed10-4268-b847-550bb644ba22.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] req_id = ddc3f64f-f47d-4189-8e15-a7b53b3bbb68
19272025-04-30T06:43:49.146ZINFOtest_utilization (BackgroundTasks): found DNS servers (initial) addresses = DnsServersList { addresses: [[::1]:42391] } background_task = dns_servers_internal dns_group = internal
19282025-04-30T06:43:49.146ZDEBGtest_utilization (BackgroundTasks): activation complete background_task = dns_servers_internal dns_group = internal elapsed = 18.461163ms iteration = 2 status = {"addresses":["[::1]:42391"]}
19292025-04-30T06:43:49.147ZINFOtest_utilization (BackgroundTasks): found DNS servers (initial) addresses = DnsServersList { addresses: [[::1]:48132] } background_task = dns_servers_external dns_group = external
19302025-04-30T06:43:49.147ZDEBGtest_utilization (BackgroundTasks): activation complete background_task = dns_servers_external dns_group = external elapsed = 17.347113ms iteration = 2 status = {"addresses":["[::1]:48132"]}
19312025-04-30T06:43:49.147ZDEBGtest_utilization (BackgroundTasks): activating background_task = dns_propagation_internal dns_group = internal iteration = 2 reason = Dependency
19322025-04-30T06:43:49.147ZWARNtest_utilization (BackgroundTasks): DNS propagation: skipped background_task = dns_propagation_internal dns_group = internal reason = no config
19332025-04-30T06:43:49.147ZDEBGtest_utilization (BackgroundTasks): activation complete background_task = dns_propagation_internal dns_group = internal elapsed = 23.066\u{b5}s iteration = 2 status = {"error":"no config"}
19342025-04-30T06:43:49.147ZDEBGtest_utilization (BackgroundTasks): activating background_task = dns_propagation_external dns_group = external iteration = 2 reason = Dependency
19352025-04-30T06:43:49.147ZWARNtest_utilization (BackgroundTasks): DNS propagation: skipped background_task = dns_propagation_external dns_group = external reason = no config
19362025-04-30T06:43:49.147ZDEBGtest_utilization (BackgroundTasks): activation complete background_task = dns_propagation_external dns_group = external elapsed = 11.533\u{b5}s iteration = 2 status = {"error":"no config"}
19372025-04-30T06:43:49.148ZDEBGtest_utilization (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
19382025-04-30T06:43:49.154ZDEBGtest_utilization (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
19392025-04-30T06:43:49.155ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database
19402025-04-30T06:43:49.155ZDEBGtest_utilization (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
19412025-04-30T06:43:49.155ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
19422025-04-30T06:43:49.156ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
19432025-04-30T06:43:49.164ZDEBGtest_utilization (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory result = Ok(())
19442025-04-30T06:43:49.164ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
19452025-04-30T06:43:49.165ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
19462025-04-30T06:43:49.167ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
19472025-04-30T06:43:49.169ZDEBGtest_utilization (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
19482025-04-30T06:43:49.183ZDEBGtest_utilization (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
19492025-04-30T06:43:49.185ZDEBGtest_utilization (BackgroundTasks): found DNS version background_task = dns_config_internal dns_group = internal version = 1
19502025-04-30T06:43:49.186ZDEBGtest_utilization (BackgroundTasks): reading DNS config background_task = dns_config_internal dns_group = internal
19512025-04-30T06:43:49.186ZDEBGtest_utilization (BackgroundTasks): reading DNS zones background_task = dns_config_internal dns_group = internal
19522025-04-30T06:43:49.186ZTRACtest_utilization (BackgroundTasks): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig
19532025-04-30T06:43:49.186ZTRACtest_utilization (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = dns_config_internal dns_group = internal resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
19542025-04-30T06:43:49.186ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database
19552025-04-30T06:43:49.186ZDEBGtest_utilization (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
19562025-04-30T06:43:49.187ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
19572025-04-30T06:43:49.188ZDEBGtest_utilization (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
19582025-04-30T06:43:49.197ZDEBGtest_utilization (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
19592025-04-30T06:43:49.201ZTRACtest_utilization (BackgroundTasks): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = DnsConfig
19602025-04-30T06:43:49.201ZTRACtest_utilization (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = external_endpoints resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
19612025-04-30T06:43:49.201ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database
19622025-04-30T06:43:49.201ZDEBGtest_utilization (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
19632025-04-30T06:43:49.202ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
19642025-04-30T06:43:49.204ZDEBGtest_utilization (BackgroundTasks): inventory_prune_one: nothing eligible for removal (too few) background_task = inventory_collection candidates = [(13d8b8d9-6c6b-47a6-9190-952544edb75b, 0)]
19652025-04-30T06:43:49.204ZTRACtest_utilization (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal
19662025-04-30T06:43:49.204ZDEBGtest_utilization (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
19672025-04-30T06:43:49.220ZDEBGtest_utilization (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
19682025-04-30T06:43:49.220ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database
19692025-04-30T06:43:49.220ZDEBGtest_utilization (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
19702025-04-30T06:43:49.221ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
19712025-04-30T06:43:49.224ZTRACtest_utilization (dns): buffer buffer = Length: 67 (0x43) bytes\n0000: 53 12 01 00 00 01 00 00 00 00 00 01 04 5f 6d 67 S............_mg\n0010: 73 04 5f 74 63 70 0d 63 6f 6e 74 72 6f 6c 2d 70 s._tcp.control-p\n0020: 6c 61 6e 65 05 6f 78 69 64 65 08 69 6e 74 65 72 lane.oxide.inter\n0030: 6e 61 6c 00 00 21 00 01 00 00 29 04 d0 00 00 00 nal..!....).....\n0040: 00 00 00 ... kind = dns peer_addr = [::1]:49416 req_id = 76327370-5403-41cf-853f-076377b43c42
19722025-04-30T06:43:49.224ZDEBGtest_utilization (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 21266,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_mgs._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_mgs._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 4,\n 95,\n 109,\n 103,\n 115,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:49416 req_id = 76327370-5403-41cf-853f-076377b43c42
19732025-04-30T06:43:49.224ZDEBGtest_utilization (store): query key key = _mgs._tcp
19742025-04-30T06:43:49.225ZDEBGtest_utilization (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
19752025-04-30T06:43:49.225ZDEBGtest_utilization (dns): dns response additional_records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal"), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] kind = dns peer_addr = [::1]:49416 query = LowerQuery { name: LowerName(Name("_mgs._tcp.control-plane.oxide.internal.")), original: Query { name: Name("_mgs._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN } } records = [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 48759, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal") })) }] req_id = 76327370-5403-41cf-853f-076377b43c42
19762025-04-30T06:43:49.225ZDEBGtest_utilization (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
19772025-04-30T06:43:49.235ZDEBGtest_utilization (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = DnsConfig result = Ok(())
19782025-04-30T06:43:49.235ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database
19792025-04-30T06:43:49.235ZDEBGtest_utilization (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
19802025-04-30T06:43:49.235ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
19812025-04-30T06:43:49.235ZDEBGtest_utilization (BackgroundTasks): found zones background_task = dns_config_internal count = 1 dns_group = internal
19822025-04-30T06:43:49.235ZDEBGtest_utilization (BackgroundTasks): listing DNS names for zone background_task = dns_config_internal batch_size = 100 dns_group = internal dns_zone_id = faf078b8-4b31-42fe-943d-bb51eaeaa558 dns_zone_name = control-plane.oxide.internal found_so_far = 0 version = 1
19832025-04-30T06:43:49.235ZTRACtest_utilization (BackgroundTasks): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig
19842025-04-30T06:43:49.235ZTRACtest_utilization (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = dns_config_internal dns_group = internal resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
19852025-04-30T06:43:49.236ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database
19862025-04-30T06:43:49.236ZDEBGtest_utilization (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
19872025-04-30T06:43:49.236ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
19882025-04-30T06:43:49.247ZDEBGtest_utilization (BackgroundTasks): found DNS version background_task = dns_config_external dns_group = external version = 2
19892025-04-30T06:43:49.247ZDEBGtest_utilization (BackgroundTasks): reading DNS config background_task = dns_config_external dns_group = external
19902025-04-30T06:43:49.248ZDEBGtest_utilization (BackgroundTasks): reading DNS zones background_task = dns_config_external dns_group = external
19912025-04-30T06:43:49.248ZTRACtest_utilization (BackgroundTasks): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig
19922025-04-30T06:43:49.248ZTRACtest_utilization (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = dns_config_external dns_group = external resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
19932025-04-30T06:43:49.248ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database
19942025-04-30T06:43:49.248ZDEBGtest_utilization (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
19952025-04-30T06:43:49.250ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
19962025-04-30T06:43:49.250ZDEBGtest_utilization (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
19972025-04-30T06:43:49.259ZDEBGtest_utilization (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
19982025-04-30T06:43:49.259ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database
19992025-04-30T06:43:49.259ZDEBGtest_utilization (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
20002025-04-30T06:43:49.260ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
20012025-04-30T06:43:49.261ZDEBGtest_utilization (ServiceBalancer): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
20022025-04-30T06:43:49.276ZDEBGtest_utilization (ServiceBalancer): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-00000000bac3, .. }) resource = Rack { parent: Fleet, key: c19a698f-c6f9-4a17-ae30-20d711b8f7dc, lookup_type: ById(c19a698f-c6f9-4a17-ae30-20d711b8f7dc) } result = Ok(())
20032025-04-30T06:43:49.277ZINFOtest_utilization (nexus): Rack initialized
20042025-04-30T06:43:49.277ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
20052025-04-30T06:43:49.277ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
20062025-04-30T06:43:49.277ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
20072025-04-30T06:43:49.278ZDEBGtest_utilization (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgs._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 48759, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }], valid_until: Instant { tv_sec: 4294, tv_nsec: 77028811 } })
20082025-04-30T06:43:49.278ZTRACtest_utilization (DnsResolver): lookup_service_targets: looking up SRV target
20092025-04-30T06:43:49.282ZDEBGtest_utilization (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
20102025-04-30T06:43:49.290ZDEBGtest_utilization (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
20112025-04-30T06:43:49.290ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database
20122025-04-30T06:43:49.290ZDEBGtest_utilization (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
20132025-04-30T06:43:49.291ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
20142025-04-30T06:43:49.291ZDEBGtest_utilization (BackgroundTasks): found all DNS names for zone background_task = dns_config_internal dns_group = internal dns_zone_id = faf078b8-4b31-42fe-943d-bb51eaeaa558 dns_zone_name = control-plane.oxide.internal found_so_far = 21 version = 1
20152025-04-30T06:43:49.292ZDEBGtest_utilization (BackgroundTasks): read DNS config background_task = dns_config_internal dns_group = internal nzones = 1 version = 1
20162025-04-30T06:43:49.292ZINFOtest_utilization (BackgroundTasks): found latest generation (first find) background_task = dns_config_internal dns_group = internal generation = 1
20172025-04-30T06:43:49.292ZDEBGtest_utilization (BackgroundTasks): activation complete background_task = dns_config_internal dns_group = internal elapsed = 166.07325ms iteration = 2 status = {"generation":1}
20182025-04-30T06:43:49.292ZDEBGtest_utilization (BackgroundTasks): activating background_task = dns_propagation_internal dns_group = internal iteration = 3 reason = Dependency
20192025-04-30T06:43:49.351ZDEBGtest_utilization (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[::1]:42391 generation = 1 method = PUT servers = DnsServersList { addresses: [[::1]:42391] } uri = http://[::1]:42391/config
20202025-04-30T06:43:49.352ZTRACtest_utilization (BackgroundTasks): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet
20212025-04-30T06:43:49.352ZTRACtest_utilization (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = external_endpoints resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
20222025-04-30T06:43:49.352ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database
20232025-04-30T06:43:49.352ZDEBGtest_utilization (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
20242025-04-30T06:43:49.354ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
20252025-04-30T06:43:49.355ZINFOtest_utilization (http): accepted connection kind = dns local_addr = [::1]:42391 remote_addr = [::1]:35059
20262025-04-30T06:43:49.356ZTRACtest_utilization (http): incoming request kind = dns local_addr = [::1]:42391 method = PUT remote_addr = [::1]:35059 req_id = 3ddb96c5-89ab-4def-96b6-cd83b5b8a0be uri = /config
20272025-04-30T06:43:49.356ZTRACtest_utilization (dns): buffer buffer = Length: 108 (0x6c) bytes\n0000: 6b 74 01 00 00 01 00 00 00 00 00 01 2d 64 65 6e kt..........-den\n0010: 64 72 69 74 65 2d 62 36 64 36 35 33 34 31 2d 31 drite-b6d65341-1\n0020: 36 37 63 2d 34 31 64 66 2d 39 62 35 63 2d 34 31 67c-41df-9b5c-41\n0030: 63 64 65 64 39 39 63 32 32 39 04 68 6f 73 74 0d cded99c229.host.\n0040: 63 6f 6e 74 72 6f 6c 2d 70 6c 61 6e 65 05 6f 78 control-plane.ox\n0050: 69 64 65 08 69 6e 74 65 72 6e 61 6c 00 00 1c 00 ide.internal....\n0060: 01 00 00 29 04 d0 00 00 00 00 00 00 ...)........ kind = dns peer_addr = [::1]:63406 req_id = f0d397ab-70c3-4d7e-a99d-89500623ae7f
20282025-04-30T06:43:49.356ZDEBGtest_utilization (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 27508,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n query_type: AAAA,\n query_class: IN,\n },\n },\n original: [\n 45,\n 100,\n 101,\n 110,\n 100,\n 114,\n 105,\n 116,\n 101,\n 45,\n 98,\n 54,\n 100,\n 54,\n 53,\n 51,\n 52,\n 49,\n 45,\n 49,\n 54,\n 55,\n 99,\n 45,\n 52,\n 49,\n 100,\n 102,\n 45,\n 57,\n 98,\n 53,\n 99,\n 45,\n 52,\n 49,\n 99,\n 100,\n 101,\n 100,\n 57,\n 57,\n 99,\n 50,\n 50,\n 57,\n 4,\n 104,\n 111,\n 115,\n 116,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 28,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:63406 req_id = f0d397ab-70c3-4d7e-a99d-89500623ae7f
20292025-04-30T06:43:49.356ZDEBGtest_utilization (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
20302025-04-30T06:43:49.356ZDEBGtest_utilization (dns): dns response additional_records = [] kind = dns peer_addr = [::1]:63406 query = LowerQuery { name: LowerName(Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.")), original: Query { name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), query_type: AAAA, query_class: IN } } records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] req_id = f0d397ab-70c3-4d7e-a99d-89500623ae7f
20312025-04-30T06:43:49.357ZINFOtest_utilization (store): attempting generation update new_generation = 1 req_id = 3ddb96c5-89ab-4def-96b6-cd83b5b8a0be
20322025-04-30T06:43:49.357ZINFOtest_utilization (store): updated generation new_generation = 1 req_id = 3ddb96c5-89ab-4def-96b6-cd83b5b8a0be
20332025-04-30T06:43:49.357ZINFOtest_utilization (http): request completed kind = dns latency_us = 1399 local_addr = [::1]:42391 method = PUT remote_addr = [::1]:35059 req_id = 3ddb96c5-89ab-4def-96b6-cd83b5b8a0be response_code = 204 uri = /config
20342025-04-30T06:43:49.357ZDEBGtest_utilization (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
20352025-04-30T06:43:49.362ZDEBGtest_utilization (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
20362025-04-30T06:43:49.362ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database
20372025-04-30T06:43:49.362ZDEBGtest_utilization (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
20382025-04-30T06:43:49.362ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
20392025-04-30T06:43:49.377ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
20402025-04-30T06:43:49.385ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
20412025-04-30T06:43:49.386ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
20422025-04-30T06:43:49.451ZTRACtest_utilization (DnsResolver): lookup_all_socket_v6 srv dns_name = _clickhouse-admin-keeper._tcp.control-plane.oxide.internal
20432025-04-30T06:43:49.452ZDEBGtest_utilization (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[::1]:42391 generation = 1 result = Ok(Response { url: "http://[::1]:42391/config", status: 204, headers: {"x-request-id": "3ddb96c5-89ab-4def-96b6-cd83b5b8a0be", "date": "Wed, 30 Apr 2025 06:43:49 GMT"} }) servers = DnsServersList { addresses: [[::1]:42391] }
20442025-04-30T06:43:49.453ZINFOtest_utilization (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[::1]:42391 generation = 1 servers = DnsServersList { addresses: [[::1]:42391] }
20452025-04-30T06:43:49.456ZINFOtest_utilization (BackgroundTasks): DNS propagation: done background_task = dns_propagation_internal dns_group = internal generation = 1 servers = DnsServersList { addresses: [[::1]:42391] }
20462025-04-30T06:43:49.458ZDEBGtest_utilization (BackgroundTasks): activation complete background_task = dns_propagation_internal dns_group = internal elapsed = 165.503485ms iteration = 3 status = {"generation":1,"server_results":{"[::1]:42391":{"Ok":null}}}
20472025-04-30T06:43:49.459ZDEBGtest_utilization (BackgroundTasks): found zones background_task = dns_config_external count = 1 dns_group = external
20482025-04-30T06:43:49.459ZDEBGtest_utilization (BackgroundTasks): listing DNS names for zone background_task = dns_config_external batch_size = 100 dns_group = external dns_zone_id = d7444068-eabc-4873-9c27-fc865773173a dns_zone_name = oxide-dev.test found_so_far = 0 version = 2
20492025-04-30T06:43:49.459ZTRACtest_utilization (BackgroundTasks): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig
20502025-04-30T06:43:49.462ZTRACtest_utilization (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = dns_config_external dns_group = external resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
20512025-04-30T06:43:49.462ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database
20522025-04-30T06:43:49.466ZDEBGtest_utilization (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
20532025-04-30T06:43:49.467ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
20542025-04-30T06:43:49.469ZTRACtest_utilization (dns): buffer buffer = Length: 87 (0x57) bytes\n0000: ca 60 01 00 00 01 00 00 00 00 00 01 18 5f 63 6c .`..........._cl\n0010: 69 63 6b 68 6f 75 73 65 2d 61 64 6d 69 6e 2d 6b ickhouse-admin-k\n0020: 65 65 70 65 72 04 5f 74 63 70 0d 63 6f 6e 74 72 eeper._tcp.contr\n0030: 6f 6c 2d 70 6c 61 6e 65 05 6f 78 69 64 65 08 69 ol-plane.oxide.i\n0040: 6e 74 65 72 6e 61 6c 00 00 21 00 01 00 00 29 04 nternal..!....).\n0050: d0 00 00 00 00 00 00 ....... kind = dns peer_addr = [::1]:58218 req_id = ac032e9b-9745-456e-b90b-41648194e845
20552025-04-30T06:43:49.470ZDEBGtest_utilization (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 51808,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 24,\n 95,\n 99,\n 108,\n 105,\n 99,\n 107,\n 104,\n 111,\n 117,\n 115,\n 101,\n 45,\n 97,\n 100,\n 109,\n 105,\n 110,\n 45,\n 107,\n 101,\n 101,\n 112,\n 101,\n 114,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:58218 req_id = ac032e9b-9745-456e-b90b-41648194e845
20562025-04-30T06:43:49.470ZDEBGtest_utilization (store): query key key = _clickhouse-admin-keeper._tcp
20572025-04-30T06:43:49.471ZERROtest_utilization (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 51808, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 1, }, query: WireQuery { query: LowerQuery { name: LowerName( Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), ), original: Query { name: Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN, }, }, original: [ 24, 95, 99, 108, 105, 99, 107, 104, 111, 117, 115, 101, 45, 97, 100, 109, 105, 110, 45, 107, 101, 101, 112, 101, 114, 4, 95, 116, 99, 112, 13, 99, 111, 110, 116, 114, 111, 108, 45, 112, 108, 97, 110, 101, 5, 111, 120, 105, 100, 101, 8, 105, 110, 116, 101, 114, 110, 97, 108, 0, 0, 33, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: Some( Edns { rcode_high: 0, version: 0, dnssec_ok: false, max_payload: 1232, options: OPT { options: {}, }, }, ), } NXDOMAIN: no records found for name: "_clickhouse-admin-keeper._tcp.control-plane.oxide.internal." kind = dns peer_addr = [::1]:58218 req_id = ac032e9b-9745-456e-b90b-41648194e845
20582025-04-30T06:43:49.472ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
20592025-04-30T06:43:49.472ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
20602025-04-30T06:43:49.473ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
20612025-04-30T06:43:49.474ZDEBGtest_utilization (BackgroundTasks): begin collection background_task = inventory_collection
20622025-04-30T06:43:49.474ZDEBGtest_utilization (BackgroundTasks): begin collection from MGS background_task = inventory_collection mgs_url = http://[::1]:48759
20632025-04-30T06:43:49.474ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/ignition
20642025-04-30T06:43:49.475ZDEBGtest_utilization (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
20652025-04-30T06:43:49.483ZDEBGtest_utilization (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
20662025-04-30T06:43:49.484ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database
20672025-04-30T06:43:49.484ZDEBGtest_utilization (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
20682025-04-30T06:43:49.484ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
20692025-04-30T06:43:49.487ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/ignition", status: 200, headers: {"content-type": "application/json", "x-request-id": "e77a04c4-cd85-4b91-83e0-ad33a8326810", "content-length": "835", "date": "Wed, 30 Apr 2025 06:43:49 GMT"} })
20702025-04-30T06:43:49.487ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0
20712025-04-30T06:43:49.488ZDEBGtest_utilization (BackgroundTasks): found all DNS names for zone background_task = dns_config_external dns_group = external dns_zone_id = d7444068-eabc-4873-9c27-fc865773173a dns_zone_name = oxide-dev.test found_so_far = 1 version = 2
20722025-04-30T06:43:49.488ZDEBGtest_utilization (BackgroundTasks): read DNS config background_task = dns_config_external dns_group = external nzones = 1 version = 2
20732025-04-30T06:43:49.488ZINFOtest_utilization (BackgroundTasks): found latest generation (first find) background_task = dns_config_external dns_group = external generation = 2
20742025-04-30T06:43:49.488ZDEBGtest_utilization (BackgroundTasks): activation complete background_task = dns_config_external dns_group = external elapsed = 360.252519ms iteration = 2 status = {"generation":2}
20752025-04-30T06:43:49.488ZDEBGtest_utilization (BackgroundTasks): activating background_task = dns_propagation_external dns_group = external iteration = 3 reason = Dependency
20762025-04-30T06:43:49.578ZDEBGtest_utilization (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[::1]:48132 generation = 2 method = PUT servers = DnsServersList { addresses: [[::1]:48132] } uri = http://[::1]:48132/config
20772025-04-30T06:43:49.578ZINFOtest_utilization (BackgroundTasks): found Silo/DNS/TLS config (initial) background_task = external_endpoints config = ExternalEndpoints { by_dns_name: {"default-silo.sys.oxide-dev.test": ExternalEndpoint { silo_id: 001de000-5110-4000-8000-000000000000, db_silo: Silo { identity: SiloIdentity { id: 001de000-5110-4000-8000-000000000000, name: Name(Name("default-silo")), description: "default silo", time_created: 2025-04-30T06:43:40.367167Z, time_modified: 2025-04-30T06:43:40.367167Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {}, rcgen: Generation(Generation(1)) }, tls_certs: [] }, "test-suite-silo.sys.oxide-dev.test": ExternalEndpoint { silo_id: ad91dcf9-5eb8-4a8d-b721-b5a47ceac136, db_silo: Silo { identity: SiloIdentity { id: ad91dcf9-5eb8-4a8d-b721-b5a47ceac136, name: Name(Name("test-suite-silo")), description: "built-in recovery Silo", time_created: 2025-04-30T06:43:48.386162Z, time_modified: 2025-04-30T06:43:48.386162Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(1)) }, tls_certs: [] }}, warnings: [NoSiloCerts { silo_id: 001de000-5110-4000-8000-000000000000, dns_name: "default-silo.sys.oxide-dev.test" }, NoSiloCerts { silo_id: ad91dcf9-5eb8-4a8d-b721-b5a47ceac136, dns_name: "test-suite-silo.sys.oxide-dev.test" }], default_endpoint: Some(ExternalEndpoint { silo_id: ad91dcf9-5eb8-4a8d-b721-b5a47ceac136, db_silo: Silo { identity: SiloIdentity { id: ad91dcf9-5eb8-4a8d-b721-b5a47ceac136, name: Name(Name("test-suite-silo")), description: "built-in recovery Silo", time_created: 2025-04-30T06:43:48.386162Z, time_modified: 2025-04-30T06:43:48.386162Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(1)) }, tls_certs: [] }) }
20782025-04-30T06:43:49.579ZDEBGtest_utilization (BackgroundTasks): activation complete background_task = external_endpoints elapsed = 449.130744ms iteration = 2 status = {"by_dns_name":{"default-silo.sys.oxide-dev.test":{"silo_id":"001de000-5110-4000-8000-000000000000","tls_certs":[]},"test-suite-silo.sys.oxide-dev.test":{"silo_id":"ad91dcf9-5eb8-4a8d-b721-b5a47ceac136","tls_certs":[]}},"default_endpoint":{"silo_id":"ad91dcf9-5eb8-4a8d-b721-b5a47ceac136","tls_certs":[]},"warnings":["silo 001de000-5110-4000-8000-000000000000 with DNS name \\"default-silo.sys.oxide-dev.test\\" has no usable certificates","silo ad91dcf9-5eb8-4a8d-b721-b5a47ceac136 with DNS name \\"test-suite-silo.sys.oxide-dev.test\\" has no usable certificates"]}
20792025-04-30T06:43:49.580ZTRACtest_utilization (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) }
20802025-04-30T06:43:49.580ZTRACtest_utilization (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-4401-4000-8000-000000000000 resource_type = Project
20812025-04-30T06:43:49.580ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
20822025-04-30T06:43:49.580ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
20832025-04-30T06:43:49.581ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
20842025-04-30T06:43:49.581ZINFOtest_utilization (http): accepted connection kind = dns local_addr = [::1]:48132 remote_addr = [::1]:57505
20852025-04-30T06:43:49.583ZTRACtest_utilization (http): incoming request kind = dns local_addr = [::1]:48132 method = PUT remote_addr = [::1]:57505 req_id = 6073e92b-bcd7-4f2e-9e72-fd39fbf42c38 uri = /config
20862025-04-30T06:43:49.584ZINFOtest_utilization (store): attempting generation update new_generation = 2 req_id = 6073e92b-bcd7-4f2e-9e72-fd39fbf42c38
20872025-04-30T06:43:49.584ZINFOtest_utilization (store): pruning trees for generations newer than 0
20882025-04-30T06:43:49.584ZDEBGtest_utilization (store): creating tree tree_name = generation_2_zone_oxide-dev.test
20892025-04-30T06:43:49.585ZDEBGtest_utilization (store): updating current config
20902025-04-30T06:43:49.585ZDEBGtest_utilization (store): flushing default tree
20912025-04-30T06:43:49.585ZINFOtest_utilization (store): pruning trees for generations older than 2 keep = 3
20922025-04-30T06:43:49.585ZINFOtest_utilization (store): updated generation new_generation = 2 req_id = 6073e92b-bcd7-4f2e-9e72-fd39fbf42c38
20932025-04-30T06:43:49.585ZINFOtest_utilization (http): request completed kind = dns latency_us = 2137 local_addr = [::1]:48132 method = PUT remote_addr = [::1]:57505 req_id = 6073e92b-bcd7-4f2e-9e72-fd39fbf42c38 response_code = 204 uri = /config
20942025-04-30T06:43:49.587ZDEBGtest_utilization (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[::1]:48132 generation = 2 result = Ok(Response { url: "http://[::1]:48132/config", status: 204, headers: {"x-request-id": "6073e92b-bcd7-4f2e-9e72-fd39fbf42c38", "date": "Wed, 30 Apr 2025 06:43:49 GMT"} }) servers = DnsServersList { addresses: [[::1]:48132] }
20952025-04-30T06:43:49.587ZINFOtest_utilization (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[::1]:48132 generation = 2 servers = DnsServersList { addresses: [[::1]:48132] }
20962025-04-30T06:43:49.597ZINFOtest_utilization (BackgroundTasks): DNS propagation: done background_task = dns_propagation_external dns_group = external generation = 2 servers = DnsServersList { addresses: [[::1]:48132] }
20972025-04-30T06:43:49.597ZDEBGtest_utilization (BackgroundTasks): activation complete background_task = dns_propagation_external dns_group = external elapsed = 108.431994ms iteration = 3 status = {"generation":2,"server_results":{"[::1]:48132":{"Ok":null}}}
20982025-04-30T06:43:49.597ZTRACtest_utilization (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-5110-4000-8000-000000000001 resource_type = Silo
20992025-04-30T06:43:49.597ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
21002025-04-30T06:43:49.597ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
21012025-04-30T06:43:49.598ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
21022025-04-30T06:43:49.613ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "a335c683-2748-4293-a922-a5c32d74c9f9", "content-length": "734", "date": "Wed, 30 Apr 2025 06:43:49 GMT"} })
21032025-04-30T06:43:49.614ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0/component/sp/caboose?firmware_slot=0
21042025-04-30T06:43:49.632ZTRACtest_utilization (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
21052025-04-30T06:43:49.632ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
21062025-04-30T06:43:49.632ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
21072025-04-30T06:43:49.633ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
21082025-04-30T06:43:49.694ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
21092025-04-30T06:43:49.707ZDEBGtest_utilization (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
21102025-04-30T06:43:49.707ZTRACtest_utilization (InternalApi): successful lookup of siloed resource "Vpc" using built-in user
21112025-04-30T06:43:49.707ZTRACtest_utilization (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) }
21122025-04-30T06:43:49.708ZTRACtest_utilization (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-4401-4000-8000-000000000000 resource_type = Project
21132025-04-30T06:43:49.708ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
21142025-04-30T06:43:49.708ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
21152025-04-30T06:43:49.708ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
21162025-04-30T06:43:49.713ZTRACtest_utilization (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-5110-4000-8000-000000000001 resource_type = Silo
21172025-04-30T06:43:49.713ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
21182025-04-30T06:43:49.713ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
21192025-04-30T06:43:49.714ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
21202025-04-30T06:43:49.716ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "09f035b4-eab7-4625-b46c-a3009d4899f7", "content-length": "111", "date": "Wed, 30 Apr 2025 06:43:49 GMT"} })
21212025-04-30T06:43:49.717ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0/component/sp/caboose?firmware_slot=1
21222025-04-30T06:43:49.718ZTRACtest_utilization (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
21232025-04-30T06:43:49.718ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
21242025-04-30T06:43:49.718ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
21252025-04-30T06:43:49.719ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
21262025-04-30T06:43:49.730ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "acb40372-b382-4e13-9097-11db3f79f3f4", "content-length": "111", "date": "Wed, 30 Apr 2025 06:43:49 GMT"} })
21272025-04-30T06:43:49.730ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0/component/rot/caboose?firmware_slot=0
21282025-04-30T06:43:49.730ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
21292025-04-30T06:43:49.767ZDEBGtest_utilization (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
21302025-04-30T06:43:49.767ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
21312025-04-30T06:43:49.767ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
21322025-04-30T06:43:49.768ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
21332025-04-30T06:43:49.771ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
21342025-04-30T06:43:49.771ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
21352025-04-30T06:43:49.772ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
21362025-04-30T06:43:49.772ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "4c3c9caf-ea2d-4a90-80aa-dc2cafa80380", "content-length": "170", "date": "Wed, 30 Apr 2025 06:43:49 GMT"} })
21372025-04-30T06:43:49.772ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0/component/rot/caboose?firmware_slot=1
21382025-04-30T06:43:49.775ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
21392025-04-30T06:43:49.775ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
21402025-04-30T06:43:49.775ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
21412025-04-30T06:43:49.776ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "47ff9bb8-3df3-4d8c-8f36-583fd0a7843f", "content-length": "170", "date": "Wed, 30 Apr 2025 06:43:49 GMT"} })
21422025-04-30T06:43:49.776ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0/component/stage0/caboose?firmware_slot=0
21432025-04-30T06:43:49.778ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
21442025-04-30T06:43:49.778ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
21452025-04-30T06:43:49.779ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
21462025-04-30T06:43:49.779ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "816be48a-3754-4f94-95e3-86c29b7dbb4c", "content-length": "179", "date": "Wed, 30 Apr 2025 06:43:49 GMT"} })
21472025-04-30T06:43:49.780ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0/component/stage0/caboose?firmware_slot=1
21482025-04-30T06:43:49.787ZTRACtest_utilization (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) }
21492025-04-30T06:43:49.787ZTRACtest_utilization (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-4401-4000-8000-000000000000 resource_type = Project
21502025-04-30T06:43:49.787ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
21512025-04-30T06:43:49.787ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
21522025-04-30T06:43:49.788ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
21532025-04-30T06:43:49.792ZTRACtest_utilization (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-5110-4000-8000-000000000001 resource_type = Silo
21542025-04-30T06:43:49.792ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
21552025-04-30T06:43:49.792ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
21562025-04-30T06:43:49.792ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
21572025-04-30T06:43:49.793ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "b53a033d-1405-40c7-b45b-d27783578fcf", "content-length": "179", "date": "Wed, 30 Apr 2025 06:43:49 GMT"} })
21582025-04-30T06:43:49.793ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0/component/rot/cmpa
21592025-04-30T06:43:49.795ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "d72a07db-1331-42f0-a686-2e1e964a7e96", "content-length": "702", "date": "Wed, 30 Apr 2025 06:43:49 GMT"} })
21602025-04-30T06:43:49.795ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0/component/rot/cfpa
21612025-04-30T06:43:49.795ZTRACtest_utilization (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
21622025-04-30T06:43:49.795ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
21632025-04-30T06:43:49.795ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
21642025-04-30T06:43:49.796ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
21652025-04-30T06:43:49.798ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "05241465-4910-4aa6-9895-7c7751c77482", "content-length": "727", "date": "Wed, 30 Apr 2025 06:43:49 GMT"} })
21662025-04-30T06:43:49.799ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0/component/rot/cfpa
21672025-04-30T06:43:49.799ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
21682025-04-30T06:43:49.807ZDEBGtest_utilization (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
21692025-04-30T06:43:49.807ZTRACtest_utilization (InternalApi): successful lookup of siloed resource "Vpc" using built-in user
21702025-04-30T06:43:49.807ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
21712025-04-30T06:43:49.807ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
21722025-04-30T06:43:49.808ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
21732025-04-30T06:43:49.811ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "277f844c-8823-42d4-aef0-5b122fbbfcd9", "content-length": "729", "date": "Wed, 30 Apr 2025 06:43:49 GMT"} })
21742025-04-30T06:43:49.811ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0/component/rot/cfpa
21752025-04-30T06:43:49.812ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
21762025-04-30T06:43:49.812ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
21772025-04-30T06:43:49.812ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
21782025-04-30T06:43:49.813ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "26d2d3ad-cdc1-4c96-b15b-47e1ab176028", "content-length": "728", "date": "Wed, 30 Apr 2025 06:43:49 GMT"} })
21792025-04-30T06:43:49.813ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1
21802025-04-30T06:43:49.816ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1", status: 200, headers: {"content-type": "application/json", "x-request-id": "82c12db8-7d93-4644-a8c0-9c8706e9b274", "content-length": "734", "date": "Wed, 30 Apr 2025 06:43:49 GMT"} })
21812025-04-30T06:43:49.816ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1/component/sp/caboose?firmware_slot=0
21822025-04-30T06:43:49.817ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
21832025-04-30T06:43:49.817ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
21842025-04-30T06:43:49.817ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
21852025-04-30T06:43:49.820ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
21862025-04-30T06:43:49.820ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
21872025-04-30T06:43:49.820ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
21882025-04-30T06:43:49.823ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "cc516374-7e06-4cce-99f8-bc5299b9a3f8", "content-length": "111", "date": "Wed, 30 Apr 2025 06:43:49 GMT"} })
21892025-04-30T06:43:49.823ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1/component/sp/caboose?firmware_slot=1
21902025-04-30T06:43:49.823ZTRACtest_utilization (InternalApi): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000002, lookup_type: ByName("nexus") }
21912025-04-30T06:43:49.823ZTRACtest_utilization (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-4401-4000-8000-000000000000 resource_type = Project
21922025-04-30T06:43:49.823ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
21932025-04-30T06:43:49.823ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
21942025-04-30T06:43:49.823ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
21952025-04-30T06:43:49.826ZTRACtest_utilization (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-5110-4000-8000-000000000001 resource_type = Silo
21962025-04-30T06:43:49.826ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
21972025-04-30T06:43:49.826ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
21982025-04-30T06:43:49.827ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
21992025-04-30T06:43:49.829ZTRACtest_utilization (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
22002025-04-30T06:43:49.829ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
22012025-04-30T06:43:49.829ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
22022025-04-30T06:43:49.829ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
22032025-04-30T06:43:49.830ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "44424224-8d83-4e47-8b45-58b96705fbe0", "content-length": "111", "date": "Wed, 30 Apr 2025 06:43:49 GMT"} })
22042025-04-30T06:43:49.830ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1/component/rot/caboose?firmware_slot=0
22052025-04-30T06:43:49.833ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
22062025-04-30T06:43:49.847ZDEBGtest_utilization (InternalApi): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000002, lookup_type: ByName("nexus") } result = Ok(())
22072025-04-30T06:43:49.847ZTRACtest_utilization (InternalApi): successful lookup of siloed resource "VpcSubnet" using built-in user
22082025-04-30T06:43:49.847ZTRACtest_utilization (InternalApi): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000002, lookup_type: ByName("nexus") }
22092025-04-30T06:43:49.847ZTRACtest_utilization (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-4401-4000-8000-000000000000 resource_type = Project
22102025-04-30T06:43:49.847ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
22112025-04-30T06:43:49.847ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
22122025-04-30T06:43:49.848ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
22132025-04-30T06:43:49.851ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "b249b3a0-701e-4130-a851-f61621fe0c01", "content-length": "170", "date": "Wed, 30 Apr 2025 06:43:49 GMT"} })
22142025-04-30T06:43:49.851ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1/component/rot/caboose?firmware_slot=1
22152025-04-30T06:43:49.851ZTRACtest_utilization (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-5110-4000-8000-000000000001 resource_type = Silo
22162025-04-30T06:43:49.851ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
22172025-04-30T06:43:49.851ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
22182025-04-30T06:43:49.852ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
22192025-04-30T06:43:49.854ZTRACtest_utilization (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
22202025-04-30T06:43:49.854ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
22212025-04-30T06:43:49.854ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
22222025-04-30T06:43:49.855ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
22232025-04-30T06:43:49.855ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "c0c4fe77-0f3f-476e-a394-45a65f814093", "content-length": "170", "date": "Wed, 30 Apr 2025 06:43:49 GMT"} })
22242025-04-30T06:43:49.855ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1/component/stage0/caboose?firmware_slot=0
22252025-04-30T06:43:49.858ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
22262025-04-30T06:43:49.865ZDEBGtest_utilization (InternalApi): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000002, lookup_type: ByName("nexus") } result = Ok(())
22272025-04-30T06:43:49.865ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
22282025-04-30T06:43:49.865ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
22292025-04-30T06:43:49.865ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
22302025-04-30T06:43:49.866ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "d4c27d6a-665c-4621-b3a0-7fad96b48ebf", "content-length": "179", "date": "Wed, 30 Apr 2025 06:43:49 GMT"} })
22312025-04-30T06:43:49.866ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1/component/stage0/caboose?firmware_slot=1
22322025-04-30T06:43:49.874ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
22332025-04-30T06:43:49.874ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
22342025-04-30T06:43:49.874ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
22352025-04-30T06:43:49.874ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "34e9bc6e-25ff-490c-9fc1-115388d63a80", "content-length": "179", "date": "Wed, 30 Apr 2025 06:43:49 GMT"} })
22362025-04-30T06:43:49.874ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1/component/rot/cmpa
22372025-04-30T06:43:49.887ZTRACtest_utilization (dns): buffer buffer = Length: 79 (0x4f) bytes\n0000: be 4a 01 00 00 01 00 00 00 00 00 01 10 5f 63 72 .J..........._cr\n0010: 75 63 69 62 6c 65 2d 70 61 6e 74 72 79 04 5f 74 ucible-pantry._t\n0020: 63 70 0d 63 6f 6e 74 72 6f 6c 2d 70 6c 61 6e 65 cp.control-plane\n0030: 05 6f 78 69 64 65 08 69 6e 74 65 72 6e 61 6c 00 .oxide.internal.\n0040: 00 21 00 01 00 00 29 04 d0 00 00 00 00 00 00 .!....)........ kind = dns peer_addr = [::1]:62228 req_id = 91ecc963-1cb5-4dbe-bb2c-06d39a94f116
22382025-04-30T06:43:49.887ZDEBGtest_utilization (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 48714,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_crucible-pantry._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_crucible-pantry._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 16,\n 95,\n 99,\n 114,\n 117,\n 99,\n 105,\n 98,\n 108,\n 101,\n 45,\n 112,\n 97,\n 110,\n 116,\n 114,\n 121,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:62228 req_id = 91ecc963-1cb5-4dbe-bb2c-06d39a94f116
22392025-04-30T06:43:49.888ZDEBGtest_utilization (store): query key key = _crucible-pantry._tcp
22402025-04-30T06:43:49.888ZDEBGtest_utilization (store): query key key = 613f608b-b7cb-4bec-8ce6-ee487de665d8.host
22412025-04-30T06:43:49.889ZDEBGtest_utilization (dns): dns response additional_records = [Record { name_labels: Name("613f608b-b7cb-4bec-8ce6-ee487de665d8.host.control-plane.oxide.internal"), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] kind = dns peer_addr = [::1]:62228 query = LowerQuery { name: LowerName(Name("_crucible-pantry._tcp.control-plane.oxide.internal.")), original: Query { name: Name("_crucible-pantry._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN } } records = [Record { name_labels: Name("_crucible-pantry._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 45558, target: Name("613f608b-b7cb-4bec-8ce6-ee487de665d8.host.control-plane.oxide.internal") })) }] req_id = 91ecc963-1cb5-4dbe-bb2c-06d39a94f116
22422025-04-30T06:43:49.891ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
22432025-04-30T06:43:49.891ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
22442025-04-30T06:43:49.892ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
22452025-04-30T06:43:49.892ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "15eb0494-fbbe-4c6a-a502-c080058f3af8", "content-length": "702", "date": "Wed, 30 Apr 2025 06:43:49 GMT"} })
22462025-04-30T06:43:49.893ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1/component/rot/cfpa
22472025-04-30T06:43:49.896ZTRACtest_utilization (dns): buffer buffer = Length: 99 (0x63) bytes\n0000: 62 4d 01 00 00 01 00 00 00 00 00 01 24 36 31 33 bM..........$613\n0010: 66 36 30 38 62 2d 62 37 63 62 2d 34 62 65 63 2d f608b-b7cb-4bec-\n0020: 38 63 65 36 2d 65 65 34 38 37 64 65 36 36 35 64 8ce6-ee487de665d\n0030: 38 04 68 6f 73 74 0d 63 6f 6e 74 72 6f 6c 2d 70 8.host.control-p\n0040: 6c 61 6e 65 05 6f 78 69 64 65 08 69 6e 74 65 72 lane.oxide.inter\n0050: 6e 61 6c 00 00 1c 00 01 00 00 29 04 d0 00 00 00 nal.......).....\n0060: 00 00 00 ... kind = dns peer_addr = [::1]:51555 req_id = 1e7ae46c-1f67-47cb-92a6-7a94c38630d1
22482025-04-30T06:43:49.896ZDEBGtest_utilization (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 25165,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("613f608b-b7cb-4bec-8ce6-ee487de665d8.host.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("613f608b-b7cb-4bec-8ce6-ee487de665d8.host.control-plane.oxide.internal."),\n query_type: AAAA,\n query_class: IN,\n },\n },\n original: [\n 36,\n 54,\n 49,\n 51,\n 102,\n 54,\n 48,\n 56,\n 98,\n 45,\n 98,\n 55,\n 99,\n 98,\n 45,\n 52,\n 98,\n 101,\n 99,\n 45,\n 56,\n 99,\n 101,\n 54,\n 45,\n 101,\n 101,\n 52,\n 56,\n 55,\n 100,\n 101,\n 54,\n 54,\n 53,\n 100,\n 56,\n 4,\n 104,\n 111,\n 115,\n 116,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 28,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:51555 req_id = 1e7ae46c-1f67-47cb-92a6-7a94c38630d1
22492025-04-30T06:43:49.897ZDEBGtest_utilization (store): query key key = 613f608b-b7cb-4bec-8ce6-ee487de665d8.host
22502025-04-30T06:43:49.897ZDEBGtest_utilization (dns): dns response additional_records = [] kind = dns peer_addr = [::1]:51555 query = LowerQuery { name: LowerName(Name("613f608b-b7cb-4bec-8ce6-ee487de665d8.host.control-plane.oxide.internal.")), original: Query { name: Name("613f608b-b7cb-4bec-8ce6-ee487de665d8.host.control-plane.oxide.internal."), query_type: AAAA, query_class: IN } } records = [Record { name_labels: Name("613f608b-b7cb-4bec-8ce6-ee487de665d8.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] req_id = 1e7ae46c-1f67-47cb-92a6-7a94c38630d1
22512025-04-30T06:43:49.946ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
22522025-04-30T06:43:49.946ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
22532025-04-30T06:43:49.947ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
22542025-04-30T06:43:49.950ZINFOtest_utilization (dropshot): accepted connection kind = pantry local_addr = [::1]:45558 remote_addr = [::1]:63203 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
22552025-04-30T06:43:49.950ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "230c3aaf-1f88-45fc-892b-acec5d32055a", "content-length": "727", "date": "Wed, 30 Apr 2025 06:43:49 GMT"} })
22562025-04-30T06:43:49.950ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1/component/rot/cfpa
22572025-04-30T06:43:49.951ZTRACtest_utilization (dropshot): incoming request kind = pantry local_addr = [::1]:45558 method = GET remote_addr = [::1]:63203 req_id = 5db4f4ee-fe99-4c3d-a201-408e641c1913 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /crucible/pantry/0
22582025-04-30T06:43:49.952ZINFOtest_utilization (dropshot): request completed kind = pantry latency_us = 857 local_addr = [::1]:45558 method = GET remote_addr = [::1]:63203 req_id = 5db4f4ee-fe99-4c3d-a201-408e641c1913 response_code = 200 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /crucible/pantry/0
22592025-04-30T06:43:50.441ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
22602025-04-30T06:43:50.441ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
22612025-04-30T06:43:50.442ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
22622025-04-30T06:43:50.443ZINFOtest_utilization (dropshot): accepted connection kind = pantry local_addr = [::1]:45558 remote_addr = [::1]:43274 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
22632025-04-30T06:43:50.443ZINFOtest_utilization (dropshot): accepted connection kind = pantry local_addr = [::1]:45558 remote_addr = [::1]:37873 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
22642025-04-30T06:43:50.443ZINFOtest_utilization (dropshot): accepted connection kind = pantry local_addr = [::1]:45558 remote_addr = [::1]:40924 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
22652025-04-30T06:43:50.443ZINFOtest_utilization (dropshot): accepted connection kind = pantry local_addr = [::1]:45558 remote_addr = [::1]:37320 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
22662025-04-30T06:43:50.443ZINFOtest_utilization (dropshot): accepted connection kind = pantry local_addr = [::1]:45558 remote_addr = [::1]:55835 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
22672025-04-30T06:43:50.443ZINFOtest_utilization (dropshot): accepted connection kind = pantry local_addr = [::1]:45558 remote_addr = [::1]:55521 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
22682025-04-30T06:43:50.443ZINFOtest_utilization (dropshot): accepted connection kind = pantry local_addr = [::1]:45558 remote_addr = [::1]:50289 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
22692025-04-30T06:43:50.452ZTRACtest_utilization (dropshot): incoming request kind = pantry local_addr = [::1]:45558 method = GET remote_addr = [::1]:43274 req_id = 4e952350-a0bb-42e8-b04d-827ae9547fe7 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /crucible/pantry/0
22702025-04-30T06:43:50.452ZTRACtest_utilization (dropshot): incoming request kind = pantry local_addr = [::1]:45558 method = GET remote_addr = [::1]:37873 req_id = f88159e9-4503-45df-b790-6f919b82d7ff sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /crucible/pantry/0
22712025-04-30T06:43:50.452ZTRACtest_utilization (dropshot): incoming request kind = pantry local_addr = [::1]:45558 method = GET remote_addr = [::1]:40924 req_id = dfda9bda-f40c-45cf-b24c-f3cd0d23e9f8 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /crucible/pantry/0
22722025-04-30T06:43:50.452ZTRACtest_utilization (dropshot): incoming request kind = pantry local_addr = [::1]:45558 method = GET remote_addr = [::1]:37320 req_id = 0be0027a-57d5-48f3-ad08-e9f23d701aff sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /crucible/pantry/0
22732025-04-30T06:43:50.454ZTRACtest_utilization (dropshot): incoming request kind = pantry local_addr = [::1]:45558 method = GET remote_addr = [::1]:55835 req_id = a95b012a-74d7-4cac-8592-aacbd070bd70 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /crucible/pantry/0
22742025-04-30T06:43:50.454ZTRACtest_utilization (dropshot): incoming request kind = pantry local_addr = [::1]:45558 method = GET remote_addr = [::1]:55521 req_id = 3b427615-13a2-43a3-984b-cf7a62ac678f sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /crucible/pantry/0
22752025-04-30T06:43:50.455ZTRACtest_utilization (dropshot): incoming request kind = pantry local_addr = [::1]:45558 method = GET remote_addr = [::1]:50289 req_id = 02ec19fe-113c-4649-9c60-120a2a1d8261 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /crucible/pantry/0
22762025-04-30T06:43:50.455ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "3295a017-551d-43de-af4d-ae8da36809cb", "content-length": "729", "date": "Wed, 30 Apr 2025 06:43:50 GMT"} })
22772025-04-30T06:43:50.455ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1/component/rot/cfpa
22782025-04-30T06:43:50.456ZINFOtest_utilization (dropshot): request completed kind = pantry latency_us = 3664 local_addr = [::1]:45558 method = GET remote_addr = [::1]:43274 req_id = 4e952350-a0bb-42e8-b04d-827ae9547fe7 response_code = 200 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /crucible/pantry/0
22792025-04-30T06:43:50.456ZINFOtest_utilization (dropshot): request completed kind = pantry latency_us = 3587 local_addr = [::1]:45558 method = GET remote_addr = [::1]:37873 req_id = f88159e9-4503-45df-b790-6f919b82d7ff response_code = 200 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /crucible/pantry/0
22802025-04-30T06:43:50.456ZINFOtest_utilization (dropshot): request completed kind = pantry latency_us = 3512 local_addr = [::1]:45558 method = GET remote_addr = [::1]:40924 req_id = dfda9bda-f40c-45cf-b24c-f3cd0d23e9f8 response_code = 200 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /crucible/pantry/0
22812025-04-30T06:43:50.456ZINFOtest_utilization (dropshot): request completed kind = pantry latency_us = 3378 local_addr = [::1]:45558 method = GET remote_addr = [::1]:37320 req_id = 0be0027a-57d5-48f3-ad08-e9f23d701aff response_code = 200 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /crucible/pantry/0
22822025-04-30T06:43:50.456ZINFOtest_utilization (dropshot): request completed kind = pantry latency_us = 1675 local_addr = [::1]:45558 method = GET remote_addr = [::1]:55835 req_id = a95b012a-74d7-4cac-8592-aacbd070bd70 response_code = 200 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /crucible/pantry/0
22832025-04-30T06:43:50.456ZINFOtest_utilization (dropshot): request completed kind = pantry latency_us = 1577 local_addr = [::1]:45558 method = GET remote_addr = [::1]:55521 req_id = 3b427615-13a2-43a3-984b-cf7a62ac678f response_code = 200 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /crucible/pantry/0
22842025-04-30T06:43:50.456ZINFOtest_utilization (dropshot): request completed kind = pantry latency_us = 1282 local_addr = [::1]:45558 method = GET remote_addr = [::1]:50289 req_id = 02ec19fe-113c-4649-9c60-120a2a1d8261 response_code = 200 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /crucible/pantry/0
22852025-04-30T06:43:50.456ZTRACtest_utilization (InternalApi): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000001, lookup_type: ByName("external-dns") }
22862025-04-30T06:43:50.456ZTRACtest_utilization (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-4401-4000-8000-000000000000 resource_type = Project
22872025-04-30T06:43:50.456ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
22882025-04-30T06:43:50.456ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
22892025-04-30T06:43:50.457ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
22902025-04-30T06:43:50.460ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "7d1deeb5-f9d3-4a1a-b2d3-3689b130741d", "content-length": "728", "date": "Wed, 30 Apr 2025 06:43:50 GMT"} })
22912025-04-30T06:43:50.460ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0
22922025-04-30T06:43:50.462ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "511a0b83-e5c0-4580-bc4c-a392ea15dc7f", "content-length": "723", "date": "Wed, 30 Apr 2025 06:43:50 GMT"} })
22932025-04-30T06:43:50.462ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0/component/sp/caboose?firmware_slot=0
22942025-04-30T06:43:50.462ZTRACtest_utilization (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-5110-4000-8000-000000000001 resource_type = Silo
22952025-04-30T06:43:50.462ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
22962025-04-30T06:43:50.462ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
22972025-04-30T06:43:50.464ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
22982025-04-30T06:43:50.467ZTRACtest_utilization (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
22992025-04-30T06:43:50.467ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
23002025-04-30T06:43:50.467ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
23012025-04-30T06:43:50.467ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
23022025-04-30T06:43:50.468ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "209bbfad-cf03-420d-9230-50b5392c9fe3", "content-length": "109", "date": "Wed, 30 Apr 2025 06:43:50 GMT"} })
23032025-04-30T06:43:50.468ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0/component/sp/caboose?firmware_slot=1
23042025-04-30T06:43:50.541ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
23052025-04-30T06:43:50.553ZDEBGtest_utilization (InternalApi): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000001, lookup_type: ByName("external-dns") } result = Ok(())
23062025-04-30T06:43:50.553ZTRACtest_utilization (InternalApi): successful lookup of siloed resource "VpcSubnet" using built-in user
23072025-04-30T06:43:50.553ZTRACtest_utilization (InternalApi): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000001, lookup_type: ByName("external-dns") }
23082025-04-30T06:43:50.553ZTRACtest_utilization (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-4401-4000-8000-000000000000 resource_type = Project
23092025-04-30T06:43:50.553ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
23102025-04-30T06:43:50.553ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
23112025-04-30T06:43:50.553ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
23122025-04-30T06:43:50.557ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "402417bc-fb59-4a65-aef0-1080e0af78f3", "content-length": "109", "date": "Wed, 30 Apr 2025 06:43:50 GMT"} })
23132025-04-30T06:43:50.557ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0/component/rot/caboose?firmware_slot=0
23142025-04-30T06:43:50.558ZTRACtest_utilization (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-5110-4000-8000-000000000001 resource_type = Silo
23152025-04-30T06:43:50.558ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
23162025-04-30T06:43:50.558ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
23172025-04-30T06:43:50.558ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
23182025-04-30T06:43:50.561ZTRACtest_utilization (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
23192025-04-30T06:43:50.561ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
23202025-04-30T06:43:50.561ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
23212025-04-30T06:43:50.562ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
23222025-04-30T06:43:50.564ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "14fb2700-642d-492a-878f-953e7a5613eb", "content-length": "169", "date": "Wed, 30 Apr 2025 06:43:50 GMT"} })
23232025-04-30T06:43:50.564ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0/component/rot/caboose?firmware_slot=1
23242025-04-30T06:43:50.564ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
23252025-04-30T06:43:50.572ZDEBGtest_utilization (InternalApi): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000001, lookup_type: ByName("external-dns") } result = Ok(())
23262025-04-30T06:43:50.573ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
23272025-04-30T06:43:50.573ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
23282025-04-30T06:43:50.573ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
23292025-04-30T06:43:50.584ZDEBGtest_utilization (InternalApi): resolved names for firewall rules instance_interfaces = {} subnet_interfaces = {Name("external-dns"): [NetworkInterface { id: e9a61db4-ad4d-462b-b2bd-03be871e2ce1, kind: Service { id: 47c428f0-ed10-4268-b847-550bb644ba22 }, name: Name("external-dns-47c428f0-ed10-4268-b847-550bb644ba22"), ip: 172.30.1.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 128, 0])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], Name("nexus"): [NetworkInterface { id: 46961afd-00ab-4b86-9fd5-30529489e593, kind: Service { id: ac55d8d9-a298-4534-898b-beee1c9e4e52 }, name: Name("nexus-ac55d8d9-a298-4534-898b-beee1c9e4e52"), ip: 172.30.2.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 128, 1])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }]} subnet_networks = {Name("nexus"): [V4(Ipv4Network { addr: 172.30.2.0, prefix: 24 }), V6(Ipv6Network { addr: fd77:e9d2:9cd9:2::, prefix: 64 })], Name("external-dns"): [V4(Ipv4Network { addr: 172.30.1.0, prefix: 24 }), V6(Ipv6Network { addr: fd77:e9d2:9cd9:1::, prefix: 64 })]} vpc_interfaces = {}
23302025-04-30T06:43:50.584ZTRACtest_utilization (InternalApi): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Fleet
23312025-04-30T06:43:50.584ZTRACtest_utilization (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
23322025-04-30T06:43:50.584ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
23332025-04-30T06:43:50.584ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
23342025-04-30T06:43:50.584ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
23352025-04-30T06:43:50.605ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
23362025-04-30T06:43:50.609ZDEBGtest_utilization (InternalApi): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Fleet result = Ok(())
23372025-04-30T06:43:50.609ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
23382025-04-30T06:43:50.609ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
23392025-04-30T06:43:50.609ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
23402025-04-30T06:43:50.613ZTRACtest_utilization (InternalApi): fetched allowlist from DB allowed = AllowList { id: 001de000-a110-4000-8000-000000000000, time_created: 2025-04-30T06:06:08.248500Z, time_modified: 2025-04-30T06:43:48.522049Z, allowed_ips: None }
23412025-04-30T06:43:50.613ZDEBGtest_utilization (InternalApi): Allowlist for user-facing services is set to allow any inbound traffic. Existing VPC firewall rules will not be modified.
23422025-04-30T06:43:50.614ZDEBGtest_utilization (InternalApi): resolved firewall rules for sled agents sled_agent_rules = [ResolvedVpcFirewallRule { action: Allow, direction: Inbound, filter_hosts: None, filter_ports: Some([L4PortRange("53")]), filter_protocols: Some([Udp]), priority: 65534, status: Enabled, targets: [NetworkInterface { id: e9a61db4-ad4d-462b-b2bd-03be871e2ce1, kind: Service { id: 47c428f0-ed10-4268-b847-550bb644ba22 }, name: Name("external-dns-47c428f0-ed10-4268-b847-550bb644ba22"), ip: 172.30.1.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 128, 0])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }] }, ResolvedVpcFirewallRule { action: Allow, direction: Inbound, filter_hosts: None, filter_ports: Some([L4PortRange("80"), L4PortRange("443")]), filter_protocols: Some([Tcp]), priority: 65534, status: Enabled, targets: [NetworkInterface { id: 46961afd-00ab-4b86-9fd5-30529489e593, kind: Service { id: ac55d8d9-a298-4534-898b-beee1c9e4e52 }, name: Name("nexus-ac55d8d9-a298-4534-898b-beee1c9e4e52"), ip: 172.30.2.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 128, 1])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }] }]
23432025-04-30T06:43:50.614ZDEBGtest_utilization (InternalApi): resolved 2 rules for sleds
23442025-04-30T06:43:50.618ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "c20f297c-980c-40aa-ad81-f901b1788858", "content-length": "169", "date": "Wed, 30 Apr 2025 06:43:50 GMT"} })
23452025-04-30T06:43:50.618ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0/component/stage0/caboose?firmware_slot=0
23462025-04-30T06:43:50.628ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "955eab0e-de66-4229-b656-464fad976ab4", "content-length": "178", "date": "Wed, 30 Apr 2025 06:43:50 GMT"} })
23472025-04-30T06:43:50.629ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0/component/stage0/caboose?firmware_slot=1
23482025-04-30T06:43:50.640ZDEBGtest_utilization (InternalApi): resolved sleds for vpc oxide-services vpc_to_sled = [Sled { identity: SledIdentity { id: b6d65341-167c-41df-9b5c-41cded99c229, time_created: 2025-04-30T06:43:42.325143Z, time_modified: 2025-04-30T06:43:42.325143Z }, time_deleted: None, rcgen: Generation(Generation(1)), rack_id: c19a698f-c6f9-4a17-ae30-20d711b8f7dc, is_scrimlet: true, serial_number: "sim-b6d65341", part_number: "Unknown", revision: SqlU32(0), usable_hardware_threads: SqlU32(16), usable_physical_ram: ByteCount(ByteCount(34359738368)), reservoir_size: ByteCount(ByteCount(17179869184)), ip: ::1, port: SqlU16(39365), last_used_address: ::ffff, policy: InService, state: Active, sled_agent_gen: Generation(Generation(1)), repo_depot_port: SqlU16(51171) }]
23492025-04-30T06:43:50.640ZDEBGtest_utilization (InternalApi): sending firewall rules to sled agents
23502025-04-30T06:43:50.640ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
23512025-04-30T06:43:50.640ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
23522025-04-30T06:43:50.641ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
23532025-04-30T06:43:50.649ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "51ace2e6-edbd-41ef-8cad-66047a560bbb", "content-length": "178", "date": "Wed, 30 Apr 2025 06:43:50 GMT"} })
23542025-04-30T06:43:50.650ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0/component/rot/cmpa
23552025-04-30T06:43:50.650ZTRACtest_utilization (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Sled { parent: Fleet, key: b6d65341-167c-41df-9b5c-41cded99c229, lookup_type: ById(b6d65341-167c-41df-9b5c-41cded99c229) }
23562025-04-30T06:43:50.650ZTRACtest_utilization (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
23572025-04-30T06:43:50.650ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
23582025-04-30T06:43:50.650ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
23592025-04-30T06:43:50.651ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
23602025-04-30T06:43:50.653ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "15b5cbb3-4c5a-4b35-be55-fa3c9f3cc5a7", "content-length": "702", "date": "Wed, 30 Apr 2025 06:43:50 GMT"} })
23612025-04-30T06:43:50.654ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0/component/rot/cfpa
23622025-04-30T06:43:50.654ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
23632025-04-30T06:43:50.660ZDEBGtest_utilization (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Sled { parent: Fleet, key: b6d65341-167c-41df-9b5c-41cded99c229, lookup_type: ById(b6d65341-167c-41df-9b5c-41cded99c229) } result = Ok(())
23642025-04-30T06:43:50.757ZDEBGtest_utilization (InternalApi): client request SledAgent = b6d65341-167c-41df-9b5c-41cded99c229 body = Some(Body) method = PUT uri = http://[::1]:39365/vpc/001de000-074c-4000-8000-000000000000/firewall/rules
23652025-04-30T06:43:50.759ZINFOtest_utilization (dropshot): accepted connection local_addr = [::1]:39365 remote_addr = [::1]:36408 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
23662025-04-30T06:43:50.762ZTRACtest_utilization (dropshot): incoming request local_addr = [::1]:39365 method = PUT remote_addr = [::1]:36408 req_id = 3a394486-29d2-401c-8a73-09fea5d71710 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /vpc/001de000-074c-4000-8000-000000000000/firewall/rules
23672025-04-30T06:43:50.762ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "4fde9827-3f76-4379-8d2c-2442e9ebf172", "content-length": "727", "date": "Wed, 30 Apr 2025 06:43:50 GMT"} })
23682025-04-30T06:43:50.763ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0/component/rot/cfpa
23692025-04-30T06:43:50.765ZINFOtest_utilization (dropshot): request completed latency_us = 2691 local_addr = [::1]:39365 method = PUT remote_addr = [::1]:36408 req_id = 3a394486-29d2-401c-8a73-09fea5d71710 response_code = 204 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /vpc/001de000-074c-4000-8000-000000000000/firewall/rules
23702025-04-30T06:43:50.765ZDEBGtest_utilization (InternalApi): client response SledAgent = b6d65341-167c-41df-9b5c-41cded99c229 result = Ok(Response { url: "http://[::1]:39365/vpc/001de000-074c-4000-8000-000000000000/firewall/rules", status: 204, headers: {"x-request-id": "3a394486-29d2-401c-8a73-09fea5d71710", "date": "Wed, 30 Apr 2025 06:43:50 GMT"} })
23712025-04-30T06:43:50.767ZINFOtest_utilization (InternalApi): updated firewall rules on 1 sleds
23722025-04-30T06:43:50.767ZINFOtest_utilization (nexus): plumbed initial IP allowlist
23732025-04-30T06:43:50.857ZDEBGtest_utilization (dropshot_external): registered endpoint local_addr = 127.0.0.1:46348 method = GET path = / versions = all
23742025-04-30T06:43:50.857ZDEBGtest_utilization (dropshot_external): registered endpoint local_addr = 127.0.0.1:46348 method = GET path = /access versions = all
23752025-04-30T06:43:50.857ZDEBGtest_utilization (dropshot_external): registered endpoint local_addr = 127.0.0.1:46348 method = GET path = /assets/{path} versions = all
23762025-04-30T06:43:50.857ZDEBGtest_utilization (dropshot_external): registered endpoint local_addr = 127.0.0.1:46348 method = POST path = /device/auth versions = all
23772025-04-30T06:43:50.857ZDEBGtest_utilization (dropshot_external): registered endpoint local_addr = 127.0.0.1:46348 method = POST path = /device/confirm versions = all
23782025-04-30T06:43:50.858ZDEBGtest_utilization (dropshot_external): registered endpoint local_addr = 127.0.0.1:46348 method = GET path = /device/success versions = all
23792025-04-30T06:43:50.858ZDEBGtest_utilization (dropshot_external): registered endpoint local_addr = 127.0.0.1:46348 method = POST path = /device/token versions = all
23802025-04-30T06:43:50.858ZDEBGtest_utilization (dropshot_external): registered endpoint local_addr = 127.0.0.1:46348 method = GET path = /device/verify versions = all
23812025-04-30T06:43:50.858ZDEBGtest_utilization (dropshot_external): registered endpoint local_addr = 127.0.0.1:46348 method = GET path = /experimental/v1/probes versions = all
23822025-04-30T06:43:50.858ZDEBGtest_utilization (dropshot_external): registered endpoint local_addr = 127.0.0.1:46348 method = POST path = /experimental/v1/probes versions = all
23832025-04-30T06:43:50.858ZDEBGtest_utilization (dropshot_external): registered endpoint local_addr = 127.0.0.1:46348 method = DELETE path = /experimental/v1/probes/{probe} versions = all
23842025-04-30T06:43:50.858ZDEBGtest_utilization (dropshot_external): registered endpoint local_addr = 127.0.0.1:46348 method = GET path = /experimental/v1/probes/{probe} versions = all
23852025-04-30T06:43:50.858ZDEBGtest_utilization (dropshot_external): registered endpoint local_addr = 127.0.0.1:46348 method = GET path = /experimental/v1/system/support-bundles versions = all
23862025-04-30T06:43:50.858ZDEBGtest_utilization (dropshot_external): registered endpoint local_addr = 127.0.0.1:46348 method = POST path = /experimental/v1/system/support-bundles versions = all
23872025-04-30T06:43:50.858ZDEBGtest_utilization (dropshot_external): registered endpoint local_addr = 127.0.0.1:46348 method = DELETE path = /experimental/v1/system/support-bundles/{support_bundle} versions = all
23882025-04-30T06:43:50.858ZDEBGtest_utilization (dropshot_external): registered endpoint local_addr = 127.0.0.1:46348 method = GET path = /experimental/v1/system/support-bundles/{support_bundle} versions = all
23892025-04-30T06:43:50.858ZDEBGtest_utilization (dropshot_external): registered endpoint local_addr = 127.0.0.1:46348 method = GET path = /experimental/v1/system/support-bundles/{support_bundle}/download versions = all
23902025-04-30T06:43:50.858ZDEBGtest_utilization (dropshot_external): registered endpoint local_addr = 127.0.0.1:46348 method = HEAD path = /experimental/v1/system/support-bundles/{support_bundle}/download versions = all
23912025-04-30T06:43:50.858ZDEBGtest_utilization (dropshot_external): registered endpoint local_addr = 127.0.0.1:46348 method = GET path = /experimental/v1/system/support-bundles/{support_bundle}/download/{file} versions = all
23922025-04-30T06:43:50.858ZDEBGtest_utilization (dropshot_external): registered endpoint local_addr = 127.0.0.1:46348 method = HEAD path = /experimental/v1/system/support-bundles/{support_bundle}/download/{file} versions = all
23932025-04-30T06:43:50.858ZDEBGtest_utilization (dropshot_external): registered endpoint local_addr = 127.0.0.1:46348 method = GET path = /experimental/v1/system/support-bundles/{support_bundle}/index versions = all
23942025-04-30T06:43:50.858ZDEBGtest_utilization (dropshot_external): registered endpoint local_addr = 127.0.0.1:46348 method = GET path = /images versions = all
23952025-04-30T06:43:50.858ZDEBGtest_utilization (dropshot_external): registered endpoint local_addr = 127.0.0.1:46348 method = GET path = /login versions = all
23962025-04-30T06:43:50.858ZDEBGtest_utilization (dropshot_external): registered endpoint local_addr = 127.0.0.1:46348 method = GET path = /login/{silo_name}/local versions = all
23972025-04-30T06:43:50.858ZDEBGtest_utilization (dropshot_external): registered endpoint local_addr = 127.0.0.1:46348 method = GET path = /login/{silo_name}/saml/{provider_name} versions = all
23982025-04-30T06:43:50.858ZDEBGtest_utilization (dropshot_external): registered endpoint local_addr = 127.0.0.1:46348 method = POST path = /login/{silo_name}/saml/{provider_name} versions = all
23992025-04-30T06:43:50.858ZDEBGtest_utilization (dropshot_external): registered endpoint local_addr = 127.0.0.1:46348 method = GET path = /login/{silo_name}/saml/{provider_name}/redirect versions = all
24002025-04-30T06:43:50.858ZDEBGtest_utilization (dropshot_external): registered endpoint local_addr = 127.0.0.1:46348 method = GET path = /lookup/{path} versions = all
24012025-04-30T06:43:50.858ZDEBGtest_utilization (dropshot_external): registered endpoint local_addr = 127.0.0.1:46348 method = GET path = /projects/{path} versions = all
24022025-04-30T06:43:50.858ZDEBGtest_utilization (dropshot_external): registered endpoint local_addr = 127.0.0.1:46348 method = GET path = /projects-new versions = all
24032025-04-30T06:43:50.858ZDEBGtest_utilization (dropshot_external): registered endpoint local_addr = 127.0.0.1:46348 method = GET path = /settings/{path} versions = all
24042025-04-30T06:43:50.858ZDEBGtest_utilization (dropshot_external): registered endpoint local_addr = 127.0.0.1:46348 method = GET path = /system/{path} versions = all
24052025-04-30T06:43:50.858ZDEBGtest_utilization (dropshot_external): registered endpoint local_addr = 127.0.0.1:46348 method = GET path = /utilization versions = all
24062025-04-30T06:43:50.858ZDEBGtest_utilization (dropshot_external): registered endpoint local_addr = 127.0.0.1:46348 method = GET path = /v1/affinity-groups versions = all
24072025-04-30T06:43:50.858ZDEBGtest_utilization (dropshot_external): registered endpoint local_addr = 127.0.0.1:46348 method = POST path = /v1/affinity-groups versions = all
24082025-04-30T06:43:50.859ZDEBGtest_utilization (dropshot_external): registered endpoint local_addr = 127.0.0.1:46348 method = DELETE path = /v1/affinity-groups/{affinity_group} versions = all
24092025-04-30T06:43:50.859ZDEBGtest_utilization (dropshot_external): registered endpoint local_addr = 127.0.0.1:46348 method = GET path = /v1/affinity-groups/{affinity_group} versions = all
24102025-04-30T06:43:50.859ZDEBGtest_utilization (dropshot_external): registered endpoint local_addr = 127.0.0.1:46348 method = PUT path = /v1/affinity-groups/{affinity_group} versions = all
24112025-04-30T06:43:50.859ZDEBGtest_utilization (dropshot_external): registered endpoint local_addr = 127.0.0.1:46348 method = GET path = /v1/affinity-groups/{affinity_group}/members versions = all
24122025-04-30T06:43:50.859ZDEBGtest_utilization (dropshot_external): registered endpoint local_addr = 127.0.0.1:46348 method = DELETE path = /v1/affinity-groups/{affinity_group}/members/instance/{instance} versions = all
24132025-04-30T06:43:50.859ZDEBGtest_utilization (dropshot_external): registered endpoint local_addr = 127.0.0.1:46348 method = GET path = /v1/affinity-groups/{affinity_group}/members/instance/{instance} versions = all
24142025-04-30T06:43:50.859ZDEBGtest_utilization (dropshot_external): registered endpoint local_addr = 127.0.0.1:46348 method = POST path = /v1/affinity-groups/{affinity_group}/members/instance/{instance} versions = all
24152025-04-30T06:43:50.859ZDEBGtest_utilization (dropshot_external): registered endpoint local_addr = 127.0.0.1:46348 method = GET path = /v1/anti-affinity-groups versions = all
24162025-04-30T06:43:50.859ZDEBGtest_utilization (dropshot_external): registered endpoint local_addr = 127.0.0.1:46348 method = POST path = /v1/anti-affinity-groups versions = all
24172025-04-30T06:43:50.859ZDEBGtest_utilization (dropshot_external): registered endpoint local_addr = 127.0.0.1:46348 method = DELETE path = /v1/anti-affinity-groups/{anti_affinity_group} versions = all
24182025-04-30T06:43:50.859ZDEBGtest_utilization (dropshot_external): registered endpoint local_addr = 127.0.0.1:46348 method = GET path = /v1/anti-affinity-groups/{anti_affinity_group} versions = all
24192025-04-30T06:43:50.859ZDEBGtest_utilization (dropshot_external): registered endpoint local_addr = 127.0.0.1:46348 method = PUT path = /v1/anti-affinity-groups/{anti_affinity_group} versions = all
24202025-04-30T06:43:50.859ZDEBGtest_utilization (dropshot_external): registered endpoint local_addr = 127.0.0.1:46348 method = GET path = /v1/anti-affinity-groups/{anti_affinity_group}/members versions = all
24212025-04-30T06:43:50.859ZDEBGtest_utilization (dropshot_external): registered endpoint local_addr = 127.0.0.1:46348 method = DELETE path = /v1/anti-affinity-groups/{anti_affinity_group}/members/instance/{instance} versions = all
24222025-04-30T06:43:50.859ZDEBGtest_utilization (dropshot_external): registered endpoint local_addr = 127.0.0.1:46348 method = GET path = /v1/anti-affinity-groups/{anti_affinity_group}/members/instance/{instance} versions = all
24232025-04-30T06:43:50.859ZDEBGtest_utilization (dropshot_external): registered endpoint local_addr = 127.0.0.1:46348 method = POST path = /v1/anti-affinity-groups/{anti_affinity_group}/members/instance/{instance} versions = all
24242025-04-30T06:43:50.859ZDEBGtest_utilization (dropshot_external): registered endpoint local_addr = 127.0.0.1:46348 method = GET path = /v1/certificates versions = all
24252025-04-30T06:43:50.859ZDEBGtest_utilization (dropshot_external): registered endpoint local_addr = 127.0.0.1:46348 method = POST path = /v1/certificates versions = all
24262025-04-30T06:43:50.860ZDEBGtest_utilization (dropshot_external): registered endpoint local_addr = 127.0.0.1:46348 method = DELETE path = /v1/certificates/{certificate} versions = all
24272025-04-30T06:43:50.860ZDEBGtest_utilization (dropshot_external): registered endpoint local_addr = 127.0.0.1:46348 method = GET path = /v1/certificates/{certificate} versions = all
24282025-04-30T06:43:50.860ZDEBGtest_utilization (dropshot_external): registered endpoint local_addr = 127.0.0.1:46348 method = GET path = /v1/disks versions = all
24292025-04-30T06:43:50.860ZDEBGtest_utilization (dropshot_external): registered endpoint local_addr = 127.0.0.1:46348 method = POST path = /v1/disks versions = all
24302025-04-30T06:43:50.860ZDEBGtest_utilization (dropshot_external): registered endpoint local_addr = 127.0.0.1:46348 method = DELETE path = /v1/disks/{disk} versions = all
24312025-04-30T06:43:50.860ZDEBGtest_utilization (dropshot_external): registered endpoint local_addr = 127.0.0.1:46348 method = GET path = /v1/disks/{disk} versions = all
24322025-04-30T06:43:50.860ZDEBGtest_utilization (dropshot_external): registered endpoint local_addr = 127.0.0.1:46348 method = POST path = /v1/disks/{disk}/bulk-write versions = all
24332025-04-30T06:43:50.860ZDEBGtest_utilization (dropshot_external): registered endpoint local_addr = 127.0.0.1:46348 method = POST path = /v1/disks/{disk}/bulk-write-start versions = all
24342025-04-30T06:43:50.860ZDEBGtest_utilization (dropshot_external): registered endpoint local_addr = 127.0.0.1:46348 method = POST path = /v1/disks/{disk}/bulk-write-stop versions = all
24352025-04-30T06:43:50.860ZDEBGtest_utilization (dropshot_external): registered endpoint local_addr = 127.0.0.1:46348 method = POST path = /v1/disks/{disk}/finalize versions = all
24362025-04-30T06:43:50.860ZDEBGtest_utilization (dropshot_external): registered endpoint local_addr = 127.0.0.1:46348 method = GET path = /v1/disks/{disk}/metrics/{metric} versions = all
24372025-04-30T06:43:50.860ZDEBGtest_utilization (dropshot_external): registered endpoint local_addr = 127.0.0.1:46348 method = GET path = /v1/floating-ips versions = all
24382025-04-30T06:43:50.860ZDEBGtest_utilization (dropshot_external): registered endpoint local_addr = 127.0.0.1:46348 method = POST path = /v1/floating-ips versions = all
24392025-04-30T06:43:50.860ZDEBGtest_utilization (dropshot_external): registered endpoint local_addr = 127.0.0.1:46348 method = DELETE path = /v1/floating-ips/{floating_ip} versions = all
24402025-04-30T06:43:50.860ZDEBGtest_utilization (dropshot_external): registered endpoint local_addr = 127.0.0.1:46348 method = GET path = /v1/floating-ips/{floating_ip} versions = all
24412025-04-30T06:43:50.860ZDEBGtest_utilization (dropshot_external): registered endpoint local_addr = 127.0.0.1:46348 method = PUT path = /v1/floating-ips/{floating_ip} versions = all
24422025-04-30T06:43:50.860ZDEBGtest_utilization (dropshot_external): registered endpoint local_addr = 127.0.0.1:46348 method = POST path = /v1/floating-ips/{floating_ip}/attach versions = all
24432025-04-30T06:43:50.860ZDEBGtest_utilization (dropshot_external): registered endpoint local_addr = 127.0.0.1:46348 method = POST path = /v1/floating-ips/{floating_ip}/detach versions = all
24442025-04-30T06:43:50.860ZDEBGtest_utilization (dropshot_external): registered endpoint local_addr = 127.0.0.1:46348 method = GET path = /v1/groups versions = all
24452025-04-30T06:43:50.860ZDEBGtest_utilization (dropshot_external): registered endpoint local_addr = 127.0.0.1:46348 method = GET path = /v1/groups/{group_id} versions = all
24462025-04-30T06:43:50.860ZDEBGtest_utilization (dropshot_external): registered endpoint local_addr = 127.0.0.1:46348 method = GET path = /v1/images versions = all
24472025-04-30T06:43:50.860ZDEBGtest_utilization (dropshot_external): registered endpoint local_addr = 127.0.0.1:46348 method = POST path = /v1/images versions = all
24482025-04-30T06:43:50.860ZDEBGtest_utilization (dropshot_external): registered endpoint local_addr = 127.0.0.1:46348 method = DELETE path = /v1/images/{image} versions = all
24492025-04-30T06:43:50.860ZDEBGtest_utilization (dropshot_external): registered endpoint local_addr = 127.0.0.1:46348 method = GET path = /v1/images/{image} versions = all
24502025-04-30T06:43:50.860ZDEBGtest_utilization (dropshot_external): registered endpoint local_addr = 127.0.0.1:46348 method = POST path = /v1/images/{image}/demote versions = all
24512025-04-30T06:43:50.861ZDEBGtest_utilization (dropshot_external): registered endpoint local_addr = 127.0.0.1:46348 method = POST path = /v1/images/{image}/promote versions = all
24522025-04-30T06:43:50.861ZDEBGtest_utilization (dropshot_external): registered endpoint local_addr = 127.0.0.1:46348 method = GET path = /v1/instances versions = all
24532025-04-30T06:43:50.861ZDEBGtest_utilization (dropshot_external): registered endpoint local_addr = 127.0.0.1:46348 method = POST path = /v1/instances versions = all
24542025-04-30T06:43:50.861ZDEBGtest_utilization (dropshot_external): registered endpoint local_addr = 127.0.0.1:46348 method = DELETE path = /v1/instances/{instance} versions = all
24552025-04-30T06:43:50.861ZDEBGtest_utilization (dropshot_external): registered endpoint local_addr = 127.0.0.1:46348 method = GET path = /v1/instances/{instance} versions = all
24562025-04-30T06:43:50.861ZDEBGtest_utilization (dropshot_external): registered endpoint local_addr = 127.0.0.1:46348 method = PUT path = /v1/instances/{instance} versions = all
24572025-04-30T06:43:50.861ZDEBGtest_utilization (dropshot_external): registered endpoint local_addr = 127.0.0.1:46348 method = GET path = /v1/instances/{instance}/affinity-groups versions = all
24582025-04-30T06:43:50.861ZDEBGtest_utilization (dropshot_external): registered endpoint local_addr = 127.0.0.1:46348 method = GET path = /v1/instances/{instance}/anti-affinity-groups versions = all
24592025-04-30T06:43:50.861ZDEBGtest_utilization (dropshot_external): registered endpoint local_addr = 127.0.0.1:46348 method = GET path = /v1/instances/{instance}/disks versions = all
24602025-04-30T06:43:50.861ZDEBGtest_utilization (dropshot_external): registered endpoint local_addr = 127.0.0.1:46348 method = POST path = /v1/instances/{instance}/disks/attach versions = all
24612025-04-30T06:43:50.861ZDEBGtest_utilization (dropshot_external): registered endpoint local_addr = 127.0.0.1:46348 method = POST path = /v1/instances/{instance}/disks/detach versions = all
24622025-04-30T06:43:50.861ZDEBGtest_utilization (dropshot_external): registered endpoint local_addr = 127.0.0.1:46348 method = GET path = /v1/instances/{instance}/external-ips versions = all
24632025-04-30T06:43:50.861ZDEBGtest_utilization (dropshot_external): registered endpoint local_addr = 127.0.0.1:46348 method = DELETE path = /v1/instances/{instance}/external-ips/ephemeral versions = all
24642025-04-30T06:43:50.861ZDEBGtest_utilization (dropshot_external): registered endpoint local_addr = 127.0.0.1:46348 method = POST path = /v1/instances/{instance}/external-ips/ephemeral versions = all
24652025-04-30T06:43:50.861ZDEBGtest_utilization (dropshot_external): registered endpoint local_addr = 127.0.0.1:46348 method = POST path = /v1/instances/{instance}/reboot versions = all
24662025-04-30T06:43:50.861ZDEBGtest_utilization (dropshot_external): registered endpoint local_addr = 127.0.0.1:46348 method = GET path = /v1/instances/{instance}/serial-console versions = all
24672025-04-30T06:43:50.861ZDEBGtest_utilization (dropshot_external): registered endpoint local_addr = 127.0.0.1:46348 method = GET path = /v1/instances/{instance}/serial-console/stream versions = all
24682025-04-30T06:43:50.861ZDEBGtest_utilization (dropshot_external): registered endpoint local_addr = 127.0.0.1:46348 method = GET path = /v1/instances/{instance}/ssh-public-keys versions = all
24692025-04-30T06:43:50.861ZDEBGtest_utilization (dropshot_external): registered endpoint local_addr = 127.0.0.1:46348 method = POST path = /v1/instances/{instance}/start versions = all
24702025-04-30T06:43:50.861ZDEBGtest_utilization (dropshot_external): registered endpoint local_addr = 127.0.0.1:46348 method = POST path = /v1/instances/{instance}/stop versions = all
24712025-04-30T06:43:50.861ZDEBGtest_utilization (dropshot_external): registered endpoint local_addr = 127.0.0.1:46348 method = GET path = /v1/internet-gateway-ip-addresses versions = all
24722025-04-30T06:43:50.861ZDEBGtest_utilization (dropshot_external): registered endpoint local_addr = 127.0.0.1:46348 method = POST path = /v1/internet-gateway-ip-addresses versions = all
24732025-04-30T06:43:50.861ZDEBGtest_utilization (dropshot_external): registered endpoint local_addr = 127.0.0.1:46348 method = DELETE path = /v1/internet-gateway-ip-addresses/{address} versions = all
24742025-04-30T06:43:50.861ZDEBGtest_utilization (dropshot_external): registered endpoint local_addr = 127.0.0.1:46348 method = GET path = /v1/internet-gateway-ip-pools versions = all
24752025-04-30T06:43:50.861ZDEBGtest_utilization (dropshot_external): registered endpoint local_addr = 127.0.0.1:46348 method = POST path = /v1/internet-gateway-ip-pools versions = all
24762025-04-30T06:43:50.861ZDEBGtest_utilization (dropshot_external): registered endpoint local_addr = 127.0.0.1:46348 method = DELETE path = /v1/internet-gateway-ip-pools/{pool} versions = all
24772025-04-30T06:43:50.861ZDEBGtest_utilization (dropshot_external): registered endpoint local_addr = 127.0.0.1:46348 method = GET path = /v1/internet-gateways versions = all
24782025-04-30T06:43:50.861ZDEBGtest_utilization (dropshot_external): registered endpoint local_addr = 127.0.0.1:46348 method = POST path = /v1/internet-gateways versions = all
24792025-04-30T06:43:50.861ZDEBGtest_utilization (dropshot_external): registered endpoint local_addr = 127.0.0.1:46348 method = DELETE path = /v1/internet-gateways/{gateway} versions = all
24802025-04-30T06:43:50.861ZDEBGtest_utilization (dropshot_external): registered endpoint local_addr = 127.0.0.1:46348 method = GET path = /v1/internet-gateways/{gateway} versions = all
24812025-04-30T06:43:50.861ZDEBGtest_utilization (dropshot_external): registered endpoint local_addr = 127.0.0.1:46348 method = GET path = /v1/ip-pools versions = all
24822025-04-30T06:43:50.861ZDEBGtest_utilization (dropshot_external): registered endpoint local_addr = 127.0.0.1:46348 method = GET path = /v1/ip-pools/{pool} versions = all
24832025-04-30T06:43:50.861ZDEBGtest_utilization (dropshot_external): registered endpoint local_addr = 127.0.0.1:46348 method = POST path = /v1/login/{silo_name}/local versions = all
24842025-04-30T06:43:50.861ZDEBGtest_utilization (dropshot_external): registered endpoint local_addr = 127.0.0.1:46348 method = POST path = /v1/logout versions = all
24852025-04-30T06:43:50.861ZDEBGtest_utilization (dropshot_external): registered endpoint local_addr = 127.0.0.1:46348 method = GET path = /v1/me versions = all
24862025-04-30T06:43:50.862ZDEBGtest_utilization (dropshot_external): registered endpoint local_addr = 127.0.0.1:46348 method = GET path = /v1/me/groups versions = all
24872025-04-30T06:43:50.862ZDEBGtest_utilization (dropshot_external): registered endpoint local_addr = 127.0.0.1:46348 method = GET path = /v1/me/ssh-keys versions = all
24882025-04-30T06:43:50.862ZDEBGtest_utilization (dropshot_external): registered endpoint local_addr = 127.0.0.1:46348 method = POST path = /v1/me/ssh-keys versions = all
24892025-04-30T06:43:50.862ZDEBGtest_utilization (dropshot_external): registered endpoint local_addr = 127.0.0.1:46348 method = DELETE path = /v1/me/ssh-keys/{ssh_key} versions = all
24902025-04-30T06:43:50.862ZDEBGtest_utilization (dropshot_external): registered endpoint local_addr = 127.0.0.1:46348 method = GET path = /v1/me/ssh-keys/{ssh_key} versions = all
24912025-04-30T06:43:50.862ZDEBGtest_utilization (dropshot_external): registered endpoint local_addr = 127.0.0.1:46348 method = GET path = /v1/metrics/{metric_name} versions = all
24922025-04-30T06:43:50.862ZDEBGtest_utilization (dropshot_external): registered endpoint local_addr = 127.0.0.1:46348 method = GET path = /v1/network-interfaces versions = all
24932025-04-30T06:43:50.862ZDEBGtest_utilization (dropshot_external): registered endpoint local_addr = 127.0.0.1:46348 method = POST path = /v1/network-interfaces versions = all
24942025-04-30T06:43:50.862ZDEBGtest_utilization (dropshot_external): registered endpoint local_addr = 127.0.0.1:46348 method = DELETE path = /v1/network-interfaces/{interface} versions = all
24952025-04-30T06:43:50.862ZDEBGtest_utilization (dropshot_external): registered endpoint local_addr = 127.0.0.1:46348 method = GET path = /v1/network-interfaces/{interface} versions = all
24962025-04-30T06:43:50.862ZDEBGtest_utilization (dropshot_external): registered endpoint local_addr = 127.0.0.1:46348 method = PUT path = /v1/network-interfaces/{interface} versions = all
24972025-04-30T06:43:50.862ZDEBGtest_utilization (dropshot_external): registered endpoint local_addr = 127.0.0.1:46348 method = GET path = /v1/ping versions = all
24982025-04-30T06:43:50.862ZDEBGtest_utilization (dropshot_external): registered endpoint local_addr = 127.0.0.1:46348 method = GET path = /v1/policy versions = all
24992025-04-30T06:43:50.862ZDEBGtest_utilization (dropshot_external): registered endpoint local_addr = 127.0.0.1:46348 method = PUT path = /v1/policy versions = all
25002025-04-30T06:43:50.862ZDEBGtest_utilization (dropshot_external): registered endpoint local_addr = 127.0.0.1:46348 method = GET path = /v1/projects versions = all
25012025-04-30T06:43:50.862ZDEBGtest_utilization (dropshot_external): registered endpoint local_addr = 127.0.0.1:46348 method = POST path = /v1/projects versions = all
25022025-04-30T06:43:50.862ZDEBGtest_utilization (dropshot_external): registered endpoint local_addr = 127.0.0.1:46348 method = DELETE path = /v1/projects/{project} versions = all
25032025-04-30T06:43:50.862ZDEBGtest_utilization (dropshot_external): registered endpoint local_addr = 127.0.0.1:46348 method = GET path = /v1/projects/{project} versions = all
25042025-04-30T06:43:50.862ZDEBGtest_utilization (dropshot_external): registered endpoint local_addr = 127.0.0.1:46348 method = PUT path = /v1/projects/{project} versions = all
25052025-04-30T06:43:50.862ZDEBGtest_utilization (dropshot_external): registered endpoint local_addr = 127.0.0.1:46348 method = GET path = /v1/projects/{project}/policy versions = all
25062025-04-30T06:43:50.862ZDEBGtest_utilization (dropshot_external): registered endpoint local_addr = 127.0.0.1:46348 method = PUT path = /v1/projects/{project}/policy versions = all
25072025-04-30T06:43:50.862ZDEBGtest_utilization (dropshot_external): registered endpoint local_addr = 127.0.0.1:46348 method = GET path = /v1/snapshots versions = all
25082025-04-30T06:43:50.862ZDEBGtest_utilization (dropshot_external): registered endpoint local_addr = 127.0.0.1:46348 method = POST path = /v1/snapshots versions = all
25092025-04-30T06:43:50.862ZDEBGtest_utilization (dropshot_external): registered endpoint local_addr = 127.0.0.1:46348 method = DELETE path = /v1/snapshots/{snapshot} versions = all
25102025-04-30T06:43:50.862ZDEBGtest_utilization (dropshot_external): registered endpoint local_addr = 127.0.0.1:46348 method = GET path = /v1/snapshots/{snapshot} versions = all
25112025-04-30T06:43:50.862ZDEBGtest_utilization (dropshot_external): registered endpoint local_addr = 127.0.0.1:46348 method = GET path = /v1/system/hardware/disks versions = all
25122025-04-30T06:43:50.862ZDEBGtest_utilization (dropshot_external): registered endpoint local_addr = 127.0.0.1:46348 method = GET path = /v1/system/hardware/disks/{disk_id} versions = all
25132025-04-30T06:43:50.862ZDEBGtest_utilization (dropshot_external): registered endpoint local_addr = 127.0.0.1:46348 method = GET path = /v1/system/hardware/rack-switch-port/{rack_id}/{switch_location}/{port}/lldp/neighbors versions = all
25142025-04-30T06:43:50.862ZDEBGtest_utilization (dropshot_external): registered endpoint local_addr = 127.0.0.1:46348 method = GET path = /v1/system/hardware/racks versions = all
25152025-04-30T06:43:50.862ZDEBGtest_utilization (dropshot_external): registered endpoint local_addr = 127.0.0.1:46348 method = GET path = /v1/system/hardware/racks/{rack_id} versions = all
25162025-04-30T06:43:50.862ZDEBGtest_utilization (dropshot_external): registered endpoint local_addr = 127.0.0.1:46348 method = GET path = /v1/system/hardware/sleds versions = all
25172025-04-30T06:43:50.862ZDEBGtest_utilization (dropshot_external): registered endpoint local_addr = 127.0.0.1:46348 method = POST path = /v1/system/hardware/sleds versions = all
25182025-04-30T06:43:50.862ZDEBGtest_utilization (dropshot_external): registered endpoint local_addr = 127.0.0.1:46348 method = GET path = /v1/system/hardware/sleds/{sled_id} versions = all
25192025-04-30T06:43:50.862ZDEBGtest_utilization (dropshot_external): registered endpoint local_addr = 127.0.0.1:46348 method = GET path = /v1/system/hardware/sleds/{sled_id}/disks versions = all
25202025-04-30T06:43:50.862ZDEBGtest_utilization (dropshot_external): registered endpoint local_addr = 127.0.0.1:46348 method = GET path = /v1/system/hardware/sleds/{sled_id}/instances versions = all
25212025-04-30T06:43:50.862ZDEBGtest_utilization (dropshot_external): registered endpoint local_addr = 127.0.0.1:46348 method = PUT path = /v1/system/hardware/sleds/{sled_id}/provision-policy versions = all
25222025-04-30T06:43:50.862ZDEBGtest_utilization (dropshot_external): registered endpoint local_addr = 127.0.0.1:46348 method = GET path = /v1/system/hardware/sleds-uninitialized versions = all
25232025-04-30T06:43:50.862ZDEBGtest_utilization (dropshot_external): registered endpoint local_addr = 127.0.0.1:46348 method = GET path = /v1/system/hardware/switch-port versions = all
25242025-04-30T06:43:50.862ZDEBGtest_utilization (dropshot_external): registered endpoint local_addr = 127.0.0.1:46348 method = GET path = /v1/system/hardware/switch-port/{port}/lldp/config versions = all
25252025-04-30T06:43:50.862ZDEBGtest_utilization (dropshot_external): registered endpoint local_addr = 127.0.0.1:46348 method = POST path = /v1/system/hardware/switch-port/{port}/lldp/config versions = all
25262025-04-30T06:43:50.862ZDEBGtest_utilization (dropshot_external): registered endpoint local_addr = 127.0.0.1:46348 method = DELETE path = /v1/system/hardware/switch-port/{port}/settings versions = all
25272025-04-30T06:43:50.862ZDEBGtest_utilization (dropshot_external): registered endpoint local_addr = 127.0.0.1:46348 method = POST path = /v1/system/hardware/switch-port/{port}/settings versions = all
25282025-04-30T06:43:50.863ZDEBGtest_utilization (dropshot_external): registered endpoint local_addr = 127.0.0.1:46348 method = GET path = /v1/system/hardware/switch-port/{port}/status versions = all
25292025-04-30T06:43:50.863ZDEBGtest_utilization (dropshot_external): registered endpoint local_addr = 127.0.0.1:46348 method = GET path = /v1/system/hardware/switches versions = all
25302025-04-30T06:43:50.863ZDEBGtest_utilization (dropshot_external): registered endpoint local_addr = 127.0.0.1:46348 method = GET path = /v1/system/hardware/switches/{switch_id} versions = all
25312025-04-30T06:43:50.863ZDEBGtest_utilization (dropshot_external): registered endpoint local_addr = 127.0.0.1:46348 method = GET path = /v1/system/identity-providers versions = all
25322025-04-30T06:43:50.863ZDEBGtest_utilization (dropshot_external): registered endpoint local_addr = 127.0.0.1:46348 method = POST path = /v1/system/identity-providers/local/users versions = all
25332025-04-30T06:43:50.863ZDEBGtest_utilization (dropshot_external): registered endpoint local_addr = 127.0.0.1:46348 method = DELETE path = /v1/system/identity-providers/local/users/{user_id} versions = all
25342025-04-30T06:43:50.863ZDEBGtest_utilization (dropshot_external): registered endpoint local_addr = 127.0.0.1:46348 method = POST path = /v1/system/identity-providers/local/users/{user_id}/set-password versions = all
25352025-04-30T06:43:50.863ZDEBGtest_utilization (dropshot_external): registered endpoint local_addr = 127.0.0.1:46348 method = POST path = /v1/system/identity-providers/saml versions = all
25362025-04-30T06:43:50.863ZDEBGtest_utilization (dropshot_external): registered endpoint local_addr = 127.0.0.1:46348 method = GET path = /v1/system/identity-providers/saml/{provider} versions = all
25372025-04-30T06:43:50.863ZDEBGtest_utilization (dropshot_external): registered endpoint local_addr = 127.0.0.1:46348 method = GET path = /v1/system/ip-pools versions = all
25382025-04-30T06:43:50.863ZDEBGtest_utilization (dropshot_external): registered endpoint local_addr = 127.0.0.1:46348 method = POST path = /v1/system/ip-pools versions = all
25392025-04-30T06:43:50.863ZDEBGtest_utilization (dropshot_external): registered endpoint local_addr = 127.0.0.1:46348 method = DELETE path = /v1/system/ip-pools/{pool} versions = all
25402025-04-30T06:43:50.863ZDEBGtest_utilization (dropshot_external): registered endpoint local_addr = 127.0.0.1:46348 method = GET path = /v1/system/ip-pools/{pool} versions = all
25412025-04-30T06:43:50.863ZDEBGtest_utilization (dropshot_external): registered endpoint local_addr = 127.0.0.1:46348 method = PUT path = /v1/system/ip-pools/{pool} versions = all
25422025-04-30T06:43:50.863ZDEBGtest_utilization (dropshot_external): registered endpoint local_addr = 127.0.0.1:46348 method = GET path = /v1/system/ip-pools/{pool}/ranges versions = all
25432025-04-30T06:43:50.863ZDEBGtest_utilization (dropshot_external): registered endpoint local_addr = 127.0.0.1:46348 method = POST path = /v1/system/ip-pools/{pool}/ranges/add versions = all
25442025-04-30T06:43:50.863ZDEBGtest_utilization (dropshot_external): registered endpoint local_addr = 127.0.0.1:46348 method = POST path = /v1/system/ip-pools/{pool}/ranges/remove versions = all
25452025-04-30T06:43:50.863ZDEBGtest_utilization (dropshot_external): registered endpoint local_addr = 127.0.0.1:46348 method = GET path = /v1/system/ip-pools/{pool}/silos versions = all
25462025-04-30T06:43:50.863ZDEBGtest_utilization (dropshot_external): registered endpoint local_addr = 127.0.0.1:46348 method = POST path = /v1/system/ip-pools/{pool}/silos versions = all
25472025-04-30T06:43:50.863ZDEBGtest_utilization (dropshot_external): registered endpoint local_addr = 127.0.0.1:46348 method = DELETE path = /v1/system/ip-pools/{pool}/silos/{silo} versions = all
25482025-04-30T06:43:50.863ZDEBGtest_utilization (dropshot_external): registered endpoint local_addr = 127.0.0.1:46348 method = PUT path = /v1/system/ip-pools/{pool}/silos/{silo} versions = all
25492025-04-30T06:43:50.863ZDEBGtest_utilization (dropshot_external): registered endpoint local_addr = 127.0.0.1:46348 method = GET path = /v1/system/ip-pools/{pool}/utilization versions = all
25502025-04-30T06:43:50.863ZDEBGtest_utilization (dropshot_external): registered endpoint local_addr = 127.0.0.1:46348 method = GET path = /v1/system/ip-pools-service versions = all
25512025-04-30T06:43:50.863ZDEBGtest_utilization (dropshot_external): registered endpoint local_addr = 127.0.0.1:46348 method = GET path = /v1/system/ip-pools-service/ranges versions = all
25522025-04-30T06:43:50.863ZDEBGtest_utilization (dropshot_external): registered endpoint local_addr = 127.0.0.1:46348 method = POST path = /v1/system/ip-pools-service/ranges/add versions = all
25532025-04-30T06:43:50.863ZDEBGtest_utilization (dropshot_external): registered endpoint local_addr = 127.0.0.1:46348 method = POST path = /v1/system/ip-pools-service/ranges/remove versions = all
25542025-04-30T06:43:50.863ZDEBGtest_utilization (dropshot_external): registered endpoint local_addr = 127.0.0.1:46348 method = GET path = /v1/system/metrics/{metric_name} versions = all
25552025-04-30T06:43:50.863ZDEBGtest_utilization (dropshot_external): registered endpoint local_addr = 127.0.0.1:46348 method = GET path = /v1/system/networking/address-lot versions = all
25562025-04-30T06:43:50.863ZDEBGtest_utilization (dropshot_external): registered endpoint local_addr = 127.0.0.1:46348 method = POST path = /v1/system/networking/address-lot versions = all
25572025-04-30T06:43:50.863ZDEBGtest_utilization (dropshot_external): registered endpoint local_addr = 127.0.0.1:46348 method = DELETE path = /v1/system/networking/address-lot/{address_lot} versions = all
25582025-04-30T06:43:50.863ZDEBGtest_utilization (dropshot_external): registered endpoint local_addr = 127.0.0.1:46348 method = GET path = /v1/system/networking/address-lot/{address_lot}/blocks versions = all
25592025-04-30T06:43:50.863ZDEBGtest_utilization (dropshot_external): registered endpoint local_addr = 127.0.0.1:46348 method = GET path = /v1/system/networking/allow-list versions = all
25602025-04-30T06:43:50.863ZDEBGtest_utilization (dropshot_external): registered endpoint local_addr = 127.0.0.1:46348 method = PUT path = /v1/system/networking/allow-list versions = all
25612025-04-30T06:43:50.863ZDEBGtest_utilization (dropshot_external): registered endpoint local_addr = 127.0.0.1:46348 method = POST path = /v1/system/networking/bfd-disable versions = all
25622025-04-30T06:43:50.863ZDEBGtest_utilization (dropshot_external): registered endpoint local_addr = 127.0.0.1:46348 method = POST path = /v1/system/networking/bfd-enable versions = all
25632025-04-30T06:43:50.863ZDEBGtest_utilization (dropshot_external): registered endpoint local_addr = 127.0.0.1:46348 method = GET path = /v1/system/networking/bfd-status versions = all
25642025-04-30T06:43:50.863ZDEBGtest_utilization (dropshot_external): registered endpoint local_addr = 127.0.0.1:46348 method = DELETE path = /v1/system/networking/bgp versions = all
25652025-04-30T06:43:50.863ZDEBGtest_utilization (dropshot_external): registered endpoint local_addr = 127.0.0.1:46348 method = GET path = /v1/system/networking/bgp versions = all
25662025-04-30T06:43:50.863ZDEBGtest_utilization (dropshot_external): registered endpoint local_addr = 127.0.0.1:46348 method = POST path = /v1/system/networking/bgp versions = all
25672025-04-30T06:43:50.863ZDEBGtest_utilization (dropshot_external): registered endpoint local_addr = 127.0.0.1:46348 method = GET path = /v1/system/networking/bgp-announce-set versions = all
25682025-04-30T06:43:50.863ZDEBGtest_utilization (dropshot_external): registered endpoint local_addr = 127.0.0.1:46348 method = PUT path = /v1/system/networking/bgp-announce-set versions = all
25692025-04-30T06:43:50.864ZDEBGtest_utilization (dropshot_external): registered endpoint local_addr = 127.0.0.1:46348 method = DELETE path = /v1/system/networking/bgp-announce-set/{announce_set} versions = all
25702025-04-30T06:43:50.864ZDEBGtest_utilization (dropshot_external): registered endpoint local_addr = 127.0.0.1:46348 method = GET path = /v1/system/networking/bgp-announce-set/{announce_set}/announcement versions = all
25712025-04-30T06:43:50.864ZDEBGtest_utilization (dropshot_external): registered endpoint local_addr = 127.0.0.1:46348 method = GET path = /v1/system/networking/bgp-exported versions = all
25722025-04-30T06:43:50.864ZDEBGtest_utilization (dropshot_external): registered endpoint local_addr = 127.0.0.1:46348 method = GET path = /v1/system/networking/bgp-message-history versions = all
25732025-04-30T06:43:50.864ZDEBGtest_utilization (dropshot_external): registered endpoint local_addr = 127.0.0.1:46348 method = GET path = /v1/system/networking/bgp-routes-ipv4 versions = all
25742025-04-30T06:43:50.864ZDEBGtest_utilization (dropshot_external): registered endpoint local_addr = 127.0.0.1:46348 method = GET path = /v1/system/networking/bgp-status versions = all
25752025-04-30T06:43:50.864ZDEBGtest_utilization (dropshot_external): registered endpoint local_addr = 127.0.0.1:46348 method = GET path = /v1/system/networking/loopback-address versions = all
25762025-04-30T06:43:50.864ZDEBGtest_utilization (dropshot_external): registered endpoint local_addr = 127.0.0.1:46348 method = POST path = /v1/system/networking/loopback-address versions = all
25772025-04-30T06:43:50.864ZDEBGtest_utilization (dropshot_external): registered endpoint local_addr = 127.0.0.1:46348 method = DELETE path = /v1/system/networking/loopback-address/{rack_id}/{switch_location}/{address}/{subnet_mask} versions = all
25782025-04-30T06:43:50.864ZDEBGtest_utilization (dropshot_external): registered endpoint local_addr = 127.0.0.1:46348 method = DELETE path = /v1/system/networking/switch-port-settings versions = all
25792025-04-30T06:43:50.864ZDEBGtest_utilization (dropshot_external): registered endpoint local_addr = 127.0.0.1:46348 method = GET path = /v1/system/networking/switch-port-settings versions = all
25802025-04-30T06:43:50.864ZDEBGtest_utilization (dropshot_external): registered endpoint local_addr = 127.0.0.1:46348 method = POST path = /v1/system/networking/switch-port-settings versions = all
25812025-04-30T06:43:50.864ZDEBGtest_utilization (dropshot_external): registered endpoint local_addr = 127.0.0.1:46348 method = GET path = /v1/system/networking/switch-port-settings/{port} versions = all
25822025-04-30T06:43:50.864ZDEBGtest_utilization (dropshot_external): registered endpoint local_addr = 127.0.0.1:46348 method = GET path = /v1/system/policy versions = all
25832025-04-30T06:43:50.864ZDEBGtest_utilization (dropshot_external): registered endpoint local_addr = 127.0.0.1:46348 method = PUT path = /v1/system/policy versions = all
25842025-04-30T06:43:50.864ZDEBGtest_utilization (dropshot_external): registered endpoint local_addr = 127.0.0.1:46348 method = GET path = /v1/system/roles versions = all
25852025-04-30T06:43:50.864ZDEBGtest_utilization (dropshot_external): registered endpoint local_addr = 127.0.0.1:46348 method = GET path = /v1/system/roles/{role_name} versions = all
25862025-04-30T06:43:50.864ZDEBGtest_utilization (dropshot_external): registered endpoint local_addr = 127.0.0.1:46348 method = GET path = /v1/system/silo-quotas versions = all
25872025-04-30T06:43:50.864ZDEBGtest_utilization (dropshot_external): registered endpoint local_addr = 127.0.0.1:46348 method = GET path = /v1/system/silos versions = all
25882025-04-30T06:43:50.864ZDEBGtest_utilization (dropshot_external): registered endpoint local_addr = 127.0.0.1:46348 method = POST path = /v1/system/silos versions = all
25892025-04-30T06:43:50.864ZDEBGtest_utilization (dropshot_external): registered endpoint local_addr = 127.0.0.1:46348 method = DELETE path = /v1/system/silos/{silo} versions = all
25902025-04-30T06:43:50.864ZDEBGtest_utilization (dropshot_external): registered endpoint local_addr = 127.0.0.1:46348 method = GET path = /v1/system/silos/{silo} versions = all
25912025-04-30T06:43:50.864ZDEBGtest_utilization (dropshot_external): registered endpoint local_addr = 127.0.0.1:46348 method = GET path = /v1/system/silos/{silo}/ip-pools versions = all
25922025-04-30T06:43:50.864ZDEBGtest_utilization (dropshot_external): registered endpoint local_addr = 127.0.0.1:46348 method = GET path = /v1/system/silos/{silo}/policy versions = all
25932025-04-30T06:43:50.864ZDEBGtest_utilization (dropshot_external): registered endpoint local_addr = 127.0.0.1:46348 method = PUT path = /v1/system/silos/{silo}/policy versions = all
25942025-04-30T06:43:50.864ZDEBGtest_utilization (dropshot_external): registered endpoint local_addr = 127.0.0.1:46348 method = GET path = /v1/system/silos/{silo}/quotas versions = all
25952025-04-30T06:43:50.864ZDEBGtest_utilization (dropshot_external): registered endpoint local_addr = 127.0.0.1:46348 method = PUT path = /v1/system/silos/{silo}/quotas versions = all
25962025-04-30T06:43:50.864ZDEBGtest_utilization (dropshot_external): registered endpoint local_addr = 127.0.0.1:46348 method = POST path = /v1/system/timeseries/query versions = all
25972025-04-30T06:43:50.864ZDEBGtest_utilization (dropshot_external): registered endpoint local_addr = 127.0.0.1:46348 method = GET path = /v1/system/timeseries/schemas versions = all
25982025-04-30T06:43:50.864ZDEBGtest_utilization (dropshot_external): registered endpoint local_addr = 127.0.0.1:46348 method = PUT path = /v1/system/update/repository versions = all
25992025-04-30T06:43:50.864ZDEBGtest_utilization (dropshot_external): registered endpoint local_addr = 127.0.0.1:46348 method = GET path = /v1/system/update/repository/{system_version} versions = all
26002025-04-30T06:43:50.864ZDEBGtest_utilization (dropshot_external): registered endpoint local_addr = 127.0.0.1:46348 method = GET path = /v1/system/update/target-release versions = all
26012025-04-30T06:43:50.864ZDEBGtest_utilization (dropshot_external): registered endpoint local_addr = 127.0.0.1:46348 method = PUT path = /v1/system/update/target-release versions = all
26022025-04-30T06:43:50.864ZDEBGtest_utilization (dropshot_external): registered endpoint local_addr = 127.0.0.1:46348 method = GET path = /v1/system/users versions = all
26032025-04-30T06:43:50.864ZDEBGtest_utilization (dropshot_external): registered endpoint local_addr = 127.0.0.1:46348 method = GET path = /v1/system/users/{user_id} versions = all
26042025-04-30T06:43:50.864ZDEBGtest_utilization (dropshot_external): registered endpoint local_addr = 127.0.0.1:46348 method = GET path = /v1/system/users-builtin versions = all
26052025-04-30T06:43:50.864ZDEBGtest_utilization (dropshot_external): registered endpoint local_addr = 127.0.0.1:46348 method = GET path = /v1/system/users-builtin/{user} versions = all
26062025-04-30T06:43:50.864ZDEBGtest_utilization (dropshot_external): registered endpoint local_addr = 127.0.0.1:46348 method = GET path = /v1/system/utilization/silos versions = all
26072025-04-30T06:43:50.864ZDEBGtest_utilization (dropshot_external): registered endpoint local_addr = 127.0.0.1:46348 method = GET path = /v1/system/utilization/silos/{silo} versions = all
26082025-04-30T06:43:50.864ZDEBGtest_utilization (dropshot_external): registered endpoint local_addr = 127.0.0.1:46348 method = POST path = /v1/timeseries/query versions = all
26092025-04-30T06:43:50.864ZDEBGtest_utilization (dropshot_external): registered endpoint local_addr = 127.0.0.1:46348 method = GET path = /v1/users versions = all
26102025-04-30T06:43:50.864ZDEBGtest_utilization (dropshot_external): registered endpoint local_addr = 127.0.0.1:46348 method = GET path = /v1/utilization versions = all
26112025-04-30T06:43:50.864ZDEBGtest_utilization (dropshot_external): registered endpoint local_addr = 127.0.0.1:46348 method = GET path = /v1/vpc-firewall-rules versions = all
26122025-04-30T06:43:50.865ZDEBGtest_utilization (dropshot_external): registered endpoint local_addr = 127.0.0.1:46348 method = PUT path = /v1/vpc-firewall-rules versions = all
26132025-04-30T06:43:50.865ZDEBGtest_utilization (dropshot_external): registered endpoint local_addr = 127.0.0.1:46348 method = GET path = /v1/vpc-router-routes versions = all
26142025-04-30T06:43:50.865ZDEBGtest_utilization (dropshot_external): registered endpoint local_addr = 127.0.0.1:46348 method = POST path = /v1/vpc-router-routes versions = all
26152025-04-30T06:43:50.865ZDEBGtest_utilization (dropshot_external): registered endpoint local_addr = 127.0.0.1:46348 method = DELETE path = /v1/vpc-router-routes/{route} versions = all
26162025-04-30T06:43:50.865ZDEBGtest_utilization (dropshot_external): registered endpoint local_addr = 127.0.0.1:46348 method = GET path = /v1/vpc-router-routes/{route} versions = all
26172025-04-30T06:43:50.865ZDEBGtest_utilization (dropshot_external): registered endpoint local_addr = 127.0.0.1:46348 method = PUT path = /v1/vpc-router-routes/{route} versions = all
26182025-04-30T06:43:50.865ZDEBGtest_utilization (dropshot_external): registered endpoint local_addr = 127.0.0.1:46348 method = GET path = /v1/vpc-routers versions = all
26192025-04-30T06:43:50.865ZDEBGtest_utilization (dropshot_external): registered endpoint local_addr = 127.0.0.1:46348 method = POST path = /v1/vpc-routers versions = all
26202025-04-30T06:43:50.865ZDEBGtest_utilization (dropshot_external): registered endpoint local_addr = 127.0.0.1:46348 method = DELETE path = /v1/vpc-routers/{router} versions = all
26212025-04-30T06:43:50.865ZDEBGtest_utilization (dropshot_external): registered endpoint local_addr = 127.0.0.1:46348 method = GET path = /v1/vpc-routers/{router} versions = all
26222025-04-30T06:43:50.865ZDEBGtest_utilization (dropshot_external): registered endpoint local_addr = 127.0.0.1:46348 method = PUT path = /v1/vpc-routers/{router} versions = all
26232025-04-30T06:43:50.865ZDEBGtest_utilization (dropshot_external): registered endpoint local_addr = 127.0.0.1:46348 method = GET path = /v1/vpc-subnets versions = all
26242025-04-30T06:43:50.865ZDEBGtest_utilization (dropshot_external): registered endpoint local_addr = 127.0.0.1:46348 method = POST path = /v1/vpc-subnets versions = all
26252025-04-30T06:43:50.865ZDEBGtest_utilization (dropshot_external): registered endpoint local_addr = 127.0.0.1:46348 method = DELETE path = /v1/vpc-subnets/{subnet} versions = all
26262025-04-30T06:43:50.865ZDEBGtest_utilization (dropshot_external): registered endpoint local_addr = 127.0.0.1:46348 method = GET path = /v1/vpc-subnets/{subnet} versions = all
26272025-04-30T06:43:50.865ZDEBGtest_utilization (dropshot_external): registered endpoint local_addr = 127.0.0.1:46348 method = PUT path = /v1/vpc-subnets/{subnet} versions = all
26282025-04-30T06:43:50.865ZDEBGtest_utilization (dropshot_external): registered endpoint local_addr = 127.0.0.1:46348 method = GET path = /v1/vpc-subnets/{subnet}/network-interfaces versions = all
26292025-04-30T06:43:50.865ZDEBGtest_utilization (dropshot_external): registered endpoint local_addr = 127.0.0.1:46348 method = GET path = /v1/vpcs versions = all
26302025-04-30T06:43:50.865ZDEBGtest_utilization (dropshot_external): registered endpoint local_addr = 127.0.0.1:46348 method = POST path = /v1/vpcs versions = all
26312025-04-30T06:43:50.865ZDEBGtest_utilization (dropshot_external): registered endpoint local_addr = 127.0.0.1:46348 method = DELETE path = /v1/vpcs/{vpc} versions = all
26322025-04-30T06:43:50.865ZDEBGtest_utilization (dropshot_external): registered endpoint local_addr = 127.0.0.1:46348 method = GET path = /v1/vpcs/{vpc} versions = all
26332025-04-30T06:43:50.865ZDEBGtest_utilization (dropshot_external): registered endpoint local_addr = 127.0.0.1:46348 method = PUT path = /v1/vpcs/{vpc} versions = all
26342025-04-30T06:43:50.865ZDEBGtest_utilization (dropshot_external): registered endpoint local_addr = 127.0.0.1:46348 method = GET path = /v1/webhooks/deliveries versions = all
26352025-04-30T06:43:50.865ZDEBGtest_utilization (dropshot_external): registered endpoint local_addr = 127.0.0.1:46348 method = POST path = /v1/webhooks/deliveries/{event_id}/resend versions = all
26362025-04-30T06:43:50.865ZDEBGtest_utilization (dropshot_external): registered endpoint local_addr = 127.0.0.1:46348 method = GET path = /v1/webhooks/event-classes versions = all
26372025-04-30T06:43:50.865ZDEBGtest_utilization (dropshot_external): registered endpoint local_addr = 127.0.0.1:46348 method = GET path = /v1/webhooks/receivers versions = all
26382025-04-30T06:43:50.865ZDEBGtest_utilization (dropshot_external): registered endpoint local_addr = 127.0.0.1:46348 method = POST path = /v1/webhooks/receivers versions = all
26392025-04-30T06:43:50.865ZDEBGtest_utilization (dropshot_external): registered endpoint local_addr = 127.0.0.1:46348 method = DELETE path = /v1/webhooks/receivers/{receiver} versions = all
26402025-04-30T06:43:50.865ZDEBGtest_utilization (dropshot_external): registered endpoint local_addr = 127.0.0.1:46348 method = GET path = /v1/webhooks/receivers/{receiver} versions = all
26412025-04-30T06:43:50.865ZDEBGtest_utilization (dropshot_external): registered endpoint local_addr = 127.0.0.1:46348 method = PUT path = /v1/webhooks/receivers/{receiver} versions = all
26422025-04-30T06:43:50.865ZDEBGtest_utilization (dropshot_external): registered endpoint local_addr = 127.0.0.1:46348 method = POST path = /v1/webhooks/receivers/{receiver}/probe versions = all
26432025-04-30T06:43:50.865ZDEBGtest_utilization (dropshot_external): registered endpoint local_addr = 127.0.0.1:46348 method = POST path = /v1/webhooks/receivers/{receiver}/subscriptions versions = all
26442025-04-30T06:43:50.865ZDEBGtest_utilization (dropshot_external): registered endpoint local_addr = 127.0.0.1:46348 method = DELETE path = /v1/webhooks/receivers/{receiver}/subscriptions/{subscription} versions = all
26452025-04-30T06:43:50.865ZDEBGtest_utilization (dropshot_external): registered endpoint local_addr = 127.0.0.1:46348 method = GET path = /v1/webhooks/secrets versions = all
26462025-04-30T06:43:50.865ZDEBGtest_utilization (dropshot_external): registered endpoint local_addr = 127.0.0.1:46348 method = POST path = /v1/webhooks/secrets versions = all
26472025-04-30T06:43:50.865ZDEBGtest_utilization (dropshot_external): registered endpoint local_addr = 127.0.0.1:46348 method = DELETE path = /v1/webhooks/secrets/{secret_id} versions = all
26482025-04-30T06:43:50.865ZINFOtest_utilization (dropshot_external): listening local_addr = 127.0.0.1:46348
26492025-04-30T06:43:50.865ZDEBGtest_utilization (dropshot_external): successfully registered DTrace USDT probes local_addr = 127.0.0.1:46348
26502025-04-30T06:43:50.905ZDEBGtest_utilization (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:53545 method = GET path = / versions = all
26512025-04-30T06:43:50.905ZDEBGtest_utilization (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:53545 method = GET path = /access versions = all
26522025-04-30T06:43:50.905ZDEBGtest_utilization (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:53545 method = GET path = /assets/{path} versions = all
26532025-04-30T06:43:50.905ZDEBGtest_utilization (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:53545 method = POST path = /device/auth versions = all
26542025-04-30T06:43:50.905ZDEBGtest_utilization (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:53545 method = POST path = /device/confirm versions = all
26552025-04-30T06:43:50.905ZDEBGtest_utilization (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:53545 method = GET path = /device/success versions = all
26562025-04-30T06:43:50.905ZDEBGtest_utilization (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:53545 method = POST path = /device/token versions = all
26572025-04-30T06:43:50.905ZDEBGtest_utilization (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:53545 method = GET path = /device/verify versions = all
26582025-04-30T06:43:50.905ZDEBGtest_utilization (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:53545 method = GET path = /experimental/v1/probes versions = all
26592025-04-30T06:43:50.905ZDEBGtest_utilization (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:53545 method = POST path = /experimental/v1/probes versions = all
26602025-04-30T06:43:50.905ZDEBGtest_utilization (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:53545 method = DELETE path = /experimental/v1/probes/{probe} versions = all
26612025-04-30T06:43:50.905ZDEBGtest_utilization (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:53545 method = GET path = /experimental/v1/probes/{probe} versions = all
26622025-04-30T06:43:50.905ZDEBGtest_utilization (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:53545 method = GET path = /experimental/v1/system/support-bundles versions = all
26632025-04-30T06:43:50.905ZDEBGtest_utilization (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:53545 method = POST path = /experimental/v1/system/support-bundles versions = all
26642025-04-30T06:43:50.905ZDEBGtest_utilization (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:53545 method = DELETE path = /experimental/v1/system/support-bundles/{support_bundle} versions = all
26652025-04-30T06:43:50.906ZDEBGtest_utilization (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:53545 method = GET path = /experimental/v1/system/support-bundles/{support_bundle} versions = all
26662025-04-30T06:43:50.906ZDEBGtest_utilization (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:53545 method = GET path = /experimental/v1/system/support-bundles/{support_bundle}/download versions = all
26672025-04-30T06:43:50.906ZDEBGtest_utilization (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:53545 method = HEAD path = /experimental/v1/system/support-bundles/{support_bundle}/download versions = all
26682025-04-30T06:43:50.906ZDEBGtest_utilization (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:53545 method = GET path = /experimental/v1/system/support-bundles/{support_bundle}/download/{file} versions = all
26692025-04-30T06:43:50.906ZDEBGtest_utilization (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:53545 method = HEAD path = /experimental/v1/system/support-bundles/{support_bundle}/download/{file} versions = all
26702025-04-30T06:43:50.906ZDEBGtest_utilization (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:53545 method = GET path = /experimental/v1/system/support-bundles/{support_bundle}/index versions = all
26712025-04-30T06:43:50.906ZDEBGtest_utilization (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:53545 method = GET path = /images versions = all
26722025-04-30T06:43:50.906ZDEBGtest_utilization (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:53545 method = GET path = /login versions = all
26732025-04-30T06:43:50.906ZDEBGtest_utilization (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:53545 method = GET path = /login/{silo_name}/local versions = all
26742025-04-30T06:43:50.906ZDEBGtest_utilization (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:53545 method = GET path = /login/{silo_name}/saml/{provider_name} versions = all
26752025-04-30T06:43:50.906ZDEBGtest_utilization (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:53545 method = POST path = /login/{silo_name}/saml/{provider_name} versions = all
26762025-04-30T06:43:50.906ZDEBGtest_utilization (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:53545 method = GET path = /login/{silo_name}/saml/{provider_name}/redirect versions = all
26772025-04-30T06:43:50.906ZDEBGtest_utilization (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:53545 method = GET path = /lookup/{path} versions = all
26782025-04-30T06:43:50.906ZDEBGtest_utilization (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:53545 method = GET path = /projects/{path} versions = all
26792025-04-30T06:43:50.906ZDEBGtest_utilization (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:53545 method = GET path = /projects-new versions = all
26802025-04-30T06:43:50.906ZDEBGtest_utilization (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:53545 method = GET path = /settings/{path} versions = all
26812025-04-30T06:43:50.906ZDEBGtest_utilization (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:53545 method = GET path = /system/{path} versions = all
26822025-04-30T06:43:50.906ZDEBGtest_utilization (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:53545 method = GET path = /utilization versions = all
26832025-04-30T06:43:50.906ZDEBGtest_utilization (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:53545 method = GET path = /v1/affinity-groups versions = all
26842025-04-30T06:43:50.906ZDEBGtest_utilization (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:53545 method = POST path = /v1/affinity-groups versions = all
26852025-04-30T06:43:50.906ZDEBGtest_utilization (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:53545 method = DELETE path = /v1/affinity-groups/{affinity_group} versions = all
26862025-04-30T06:43:50.906ZDEBGtest_utilization (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:53545 method = GET path = /v1/affinity-groups/{affinity_group} versions = all
26872025-04-30T06:43:50.906ZDEBGtest_utilization (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:53545 method = PUT path = /v1/affinity-groups/{affinity_group} versions = all
26882025-04-30T06:43:50.906ZDEBGtest_utilization (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:53545 method = GET path = /v1/affinity-groups/{affinity_group}/members versions = all
26892025-04-30T06:43:50.906ZDEBGtest_utilization (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:53545 method = DELETE path = /v1/affinity-groups/{affinity_group}/members/instance/{instance} versions = all
26902025-04-30T06:43:50.906ZDEBGtest_utilization (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:53545 method = GET path = /v1/affinity-groups/{affinity_group}/members/instance/{instance} versions = all
26912025-04-30T06:43:50.906ZDEBGtest_utilization (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:53545 method = POST path = /v1/affinity-groups/{affinity_group}/members/instance/{instance} versions = all
26922025-04-30T06:43:50.906ZDEBGtest_utilization (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:53545 method = GET path = /v1/anti-affinity-groups versions = all
26932025-04-30T06:43:50.906ZDEBGtest_utilization (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:53545 method = POST path = /v1/anti-affinity-groups versions = all
26942025-04-30T06:43:50.906ZDEBGtest_utilization (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:53545 method = DELETE path = /v1/anti-affinity-groups/{anti_affinity_group} versions = all
26952025-04-30T06:43:50.906ZDEBGtest_utilization (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:53545 method = GET path = /v1/anti-affinity-groups/{anti_affinity_group} versions = all
26962025-04-30T06:43:50.906ZDEBGtest_utilization (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:53545 method = PUT path = /v1/anti-affinity-groups/{anti_affinity_group} versions = all
26972025-04-30T06:43:50.906ZDEBGtest_utilization (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:53545 method = GET path = /v1/anti-affinity-groups/{anti_affinity_group}/members versions = all
26982025-04-30T06:43:50.906ZDEBGtest_utilization (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:53545 method = DELETE path = /v1/anti-affinity-groups/{anti_affinity_group}/members/instance/{instance} versions = all
26992025-04-30T06:43:50.906ZDEBGtest_utilization (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:53545 method = GET path = /v1/anti-affinity-groups/{anti_affinity_group}/members/instance/{instance} versions = all
27002025-04-30T06:43:50.906ZDEBGtest_utilization (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:53545 method = POST path = /v1/anti-affinity-groups/{anti_affinity_group}/members/instance/{instance} versions = all
27012025-04-30T06:43:50.906ZDEBGtest_utilization (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:53545 method = GET path = /v1/certificates versions = all
27022025-04-30T06:43:50.907ZDEBGtest_utilization (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:53545 method = POST path = /v1/certificates versions = all
27032025-04-30T06:43:50.907ZDEBGtest_utilization (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:53545 method = DELETE path = /v1/certificates/{certificate} versions = all
27042025-04-30T06:43:50.907ZDEBGtest_utilization (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:53545 method = GET path = /v1/certificates/{certificate} versions = all
27052025-04-30T06:43:50.907ZDEBGtest_utilization (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:53545 method = GET path = /v1/disks versions = all
27062025-04-30T06:43:50.907ZDEBGtest_utilization (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:53545 method = POST path = /v1/disks versions = all
27072025-04-30T06:43:50.907ZDEBGtest_utilization (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:53545 method = DELETE path = /v1/disks/{disk} versions = all
27082025-04-30T06:43:50.907ZDEBGtest_utilization (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:53545 method = GET path = /v1/disks/{disk} versions = all
27092025-04-30T06:43:50.907ZDEBGtest_utilization (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:53545 method = POST path = /v1/disks/{disk}/bulk-write versions = all
27102025-04-30T06:43:50.907ZDEBGtest_utilization (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:53545 method = POST path = /v1/disks/{disk}/bulk-write-start versions = all
27112025-04-30T06:43:50.907ZDEBGtest_utilization (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:53545 method = POST path = /v1/disks/{disk}/bulk-write-stop versions = all
27122025-04-30T06:43:50.907ZDEBGtest_utilization (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:53545 method = POST path = /v1/disks/{disk}/finalize versions = all
27132025-04-30T06:43:50.907ZDEBGtest_utilization (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:53545 method = GET path = /v1/disks/{disk}/metrics/{metric} versions = all
27142025-04-30T06:43:50.907ZDEBGtest_utilization (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:53545 method = GET path = /v1/floating-ips versions = all
27152025-04-30T06:43:50.907ZDEBGtest_utilization (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:53545 method = POST path = /v1/floating-ips versions = all
27162025-04-30T06:43:50.907ZDEBGtest_utilization (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:53545 method = DELETE path = /v1/floating-ips/{floating_ip} versions = all
27172025-04-30T06:43:50.907ZDEBGtest_utilization (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:53545 method = GET path = /v1/floating-ips/{floating_ip} versions = all
27182025-04-30T06:43:50.907ZDEBGtest_utilization (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:53545 method = PUT path = /v1/floating-ips/{floating_ip} versions = all
27192025-04-30T06:43:50.907ZDEBGtest_utilization (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:53545 method = POST path = /v1/floating-ips/{floating_ip}/attach versions = all
27202025-04-30T06:43:50.907ZDEBGtest_utilization (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:53545 method = POST path = /v1/floating-ips/{floating_ip}/detach versions = all
27212025-04-30T06:43:50.907ZDEBGtest_utilization (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:53545 method = GET path = /v1/groups versions = all
27222025-04-30T06:43:50.907ZDEBGtest_utilization (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:53545 method = GET path = /v1/groups/{group_id} versions = all
27232025-04-30T06:43:50.907ZDEBGtest_utilization (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:53545 method = GET path = /v1/images versions = all
27242025-04-30T06:43:50.907ZDEBGtest_utilization (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:53545 method = POST path = /v1/images versions = all
27252025-04-30T06:43:50.907ZDEBGtest_utilization (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:53545 method = DELETE path = /v1/images/{image} versions = all
27262025-04-30T06:43:50.907ZDEBGtest_utilization (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:53545 method = GET path = /v1/images/{image} versions = all
27272025-04-30T06:43:50.907ZDEBGtest_utilization (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:53545 method = POST path = /v1/images/{image}/demote versions = all
27282025-04-30T06:43:50.907ZDEBGtest_utilization (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:53545 method = POST path = /v1/images/{image}/promote versions = all
27292025-04-30T06:43:50.907ZDEBGtest_utilization (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:53545 method = GET path = /v1/instances versions = all
27302025-04-30T06:43:50.907ZDEBGtest_utilization (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:53545 method = POST path = /v1/instances versions = all
27312025-04-30T06:43:50.907ZDEBGtest_utilization (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:53545 method = DELETE path = /v1/instances/{instance} versions = all
27322025-04-30T06:43:50.907ZDEBGtest_utilization (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:53545 method = GET path = /v1/instances/{instance} versions = all
27332025-04-30T06:43:50.907ZDEBGtest_utilization (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:53545 method = PUT path = /v1/instances/{instance} versions = all
27342025-04-30T06:43:50.907ZDEBGtest_utilization (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:53545 method = GET path = /v1/instances/{instance}/affinity-groups versions = all
27352025-04-30T06:43:50.907ZDEBGtest_utilization (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:53545 method = GET path = /v1/instances/{instance}/anti-affinity-groups versions = all
27362025-04-30T06:43:50.907ZDEBGtest_utilization (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:53545 method = GET path = /v1/instances/{instance}/disks versions = all
27372025-04-30T06:43:50.907ZDEBGtest_utilization (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:53545 method = POST path = /v1/instances/{instance}/disks/attach versions = all
27382025-04-30T06:43:50.907ZDEBGtest_utilization (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:53545 method = POST path = /v1/instances/{instance}/disks/detach versions = all
27392025-04-30T06:43:50.907ZDEBGtest_utilization (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:53545 method = GET path = /v1/instances/{instance}/external-ips versions = all
27402025-04-30T06:43:50.907ZDEBGtest_utilization (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:53545 method = DELETE path = /v1/instances/{instance}/external-ips/ephemeral versions = all
27412025-04-30T06:43:50.907ZDEBGtest_utilization (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:53545 method = POST path = /v1/instances/{instance}/external-ips/ephemeral versions = all
27422025-04-30T06:43:50.907ZDEBGtest_utilization (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:53545 method = POST path = /v1/instances/{instance}/reboot versions = all
27432025-04-30T06:43:50.907ZDEBGtest_utilization (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:53545 method = GET path = /v1/instances/{instance}/serial-console versions = all
27442025-04-30T06:43:50.907ZDEBGtest_utilization (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:53545 method = GET path = /v1/instances/{instance}/serial-console/stream versions = all
27452025-04-30T06:43:50.907ZDEBGtest_utilization (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:53545 method = GET path = /v1/instances/{instance}/ssh-public-keys versions = all
27462025-04-30T06:43:50.908ZDEBGtest_utilization (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:53545 method = POST path = /v1/instances/{instance}/start versions = all
27472025-04-30T06:43:50.908ZDEBGtest_utilization (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:53545 method = POST path = /v1/instances/{instance}/stop versions = all
27482025-04-30T06:43:50.908ZDEBGtest_utilization (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:53545 method = GET path = /v1/internet-gateway-ip-addresses versions = all
27492025-04-30T06:43:50.908ZDEBGtest_utilization (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:53545 method = POST path = /v1/internet-gateway-ip-addresses versions = all
27502025-04-30T06:43:50.908ZDEBGtest_utilization (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:53545 method = DELETE path = /v1/internet-gateway-ip-addresses/{address} versions = all
27512025-04-30T06:43:50.908ZDEBGtest_utilization (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:53545 method = GET path = /v1/internet-gateway-ip-pools versions = all
27522025-04-30T06:43:50.908ZDEBGtest_utilization (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:53545 method = POST path = /v1/internet-gateway-ip-pools versions = all
27532025-04-30T06:43:50.908ZDEBGtest_utilization (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:53545 method = DELETE path = /v1/internet-gateway-ip-pools/{pool} versions = all
27542025-04-30T06:43:50.908ZDEBGtest_utilization (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:53545 method = GET path = /v1/internet-gateways versions = all
27552025-04-30T06:43:50.908ZDEBGtest_utilization (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:53545 method = POST path = /v1/internet-gateways versions = all
27562025-04-30T06:43:50.908ZDEBGtest_utilization (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:53545 method = DELETE path = /v1/internet-gateways/{gateway} versions = all
27572025-04-30T06:43:50.908ZDEBGtest_utilization (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:53545 method = GET path = /v1/internet-gateways/{gateway} versions = all
27582025-04-30T06:43:50.908ZDEBGtest_utilization (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:53545 method = GET path = /v1/ip-pools versions = all
27592025-04-30T06:43:50.908ZDEBGtest_utilization (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:53545 method = GET path = /v1/ip-pools/{pool} versions = all
27602025-04-30T06:43:50.908ZDEBGtest_utilization (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:53545 method = POST path = /v1/login/{silo_name}/local versions = all
27612025-04-30T06:43:50.908ZDEBGtest_utilization (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:53545 method = POST path = /v1/logout versions = all
27622025-04-30T06:43:50.908ZDEBGtest_utilization (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:53545 method = GET path = /v1/me versions = all
27632025-04-30T06:43:50.908ZDEBGtest_utilization (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:53545 method = GET path = /v1/me/groups versions = all
27642025-04-30T06:43:50.908ZDEBGtest_utilization (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:53545 method = GET path = /v1/me/ssh-keys versions = all
27652025-04-30T06:43:50.908ZDEBGtest_utilization (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:53545 method = POST path = /v1/me/ssh-keys versions = all
27662025-04-30T06:43:50.908ZDEBGtest_utilization (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:53545 method = DELETE path = /v1/me/ssh-keys/{ssh_key} versions = all
27672025-04-30T06:43:50.908ZDEBGtest_utilization (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:53545 method = GET path = /v1/me/ssh-keys/{ssh_key} versions = all
27682025-04-30T06:43:50.908ZDEBGtest_utilization (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:53545 method = GET path = /v1/metrics/{metric_name} versions = all
27692025-04-30T06:43:50.908ZDEBGtest_utilization (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:53545 method = GET path = /v1/network-interfaces versions = all
27702025-04-30T06:43:50.908ZDEBGtest_utilization (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:53545 method = POST path = /v1/network-interfaces versions = all
27712025-04-30T06:43:50.908ZDEBGtest_utilization (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:53545 method = DELETE path = /v1/network-interfaces/{interface} versions = all
27722025-04-30T06:43:50.908ZDEBGtest_utilization (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:53545 method = GET path = /v1/network-interfaces/{interface} versions = all
27732025-04-30T06:43:50.908ZDEBGtest_utilization (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:53545 method = PUT path = /v1/network-interfaces/{interface} versions = all
27742025-04-30T06:43:50.908ZDEBGtest_utilization (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:53545 method = GET path = /v1/ping versions = all
27752025-04-30T06:43:50.908ZDEBGtest_utilization (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:53545 method = GET path = /v1/policy versions = all
27762025-04-30T06:43:50.908ZDEBGtest_utilization (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:53545 method = PUT path = /v1/policy versions = all
27772025-04-30T06:43:50.908ZDEBGtest_utilization (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:53545 method = GET path = /v1/projects versions = all
27782025-04-30T06:43:50.908ZDEBGtest_utilization (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:53545 method = POST path = /v1/projects versions = all
27792025-04-30T06:43:50.908ZDEBGtest_utilization (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:53545 method = DELETE path = /v1/projects/{project} versions = all
27802025-04-30T06:43:50.908ZDEBGtest_utilization (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:53545 method = GET path = /v1/projects/{project} versions = all
27812025-04-30T06:43:50.908ZDEBGtest_utilization (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:53545 method = PUT path = /v1/projects/{project} versions = all
27822025-04-30T06:43:50.908ZDEBGtest_utilization (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:53545 method = GET path = /v1/projects/{project}/policy versions = all
27832025-04-30T06:43:50.908ZDEBGtest_utilization (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:53545 method = PUT path = /v1/projects/{project}/policy versions = all
27842025-04-30T06:43:50.908ZDEBGtest_utilization (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:53545 method = GET path = /v1/snapshots versions = all
27852025-04-30T06:43:50.908ZDEBGtest_utilization (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:53545 method = POST path = /v1/snapshots versions = all
27862025-04-30T06:43:50.908ZDEBGtest_utilization (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:53545 method = DELETE path = /v1/snapshots/{snapshot} versions = all
27872025-04-30T06:43:50.908ZDEBGtest_utilization (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:53545 method = GET path = /v1/snapshots/{snapshot} versions = all
27882025-04-30T06:43:50.908ZDEBGtest_utilization (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:53545 method = GET path = /v1/system/hardware/disks versions = all
27892025-04-30T06:43:50.908ZDEBGtest_utilization (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:53545 method = GET path = /v1/system/hardware/disks/{disk_id} versions = all
27902025-04-30T06:43:50.909ZDEBGtest_utilization (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:53545 method = GET path = /v1/system/hardware/rack-switch-port/{rack_id}/{switch_location}/{port}/lldp/neighbors versions = all
27912025-04-30T06:43:50.909ZDEBGtest_utilization (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:53545 method = GET path = /v1/system/hardware/racks versions = all
27922025-04-30T06:43:50.909ZDEBGtest_utilization (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:53545 method = GET path = /v1/system/hardware/racks/{rack_id} versions = all
27932025-04-30T06:43:50.909ZDEBGtest_utilization (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:53545 method = GET path = /v1/system/hardware/sleds versions = all
27942025-04-30T06:43:50.909ZDEBGtest_utilization (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:53545 method = POST path = /v1/system/hardware/sleds versions = all
27952025-04-30T06:43:50.909ZDEBGtest_utilization (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:53545 method = GET path = /v1/system/hardware/sleds/{sled_id} versions = all
27962025-04-30T06:43:50.909ZDEBGtest_utilization (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:53545 method = GET path = /v1/system/hardware/sleds/{sled_id}/disks versions = all
27972025-04-30T06:43:50.909ZDEBGtest_utilization (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:53545 method = GET path = /v1/system/hardware/sleds/{sled_id}/instances versions = all
27982025-04-30T06:43:50.909ZDEBGtest_utilization (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:53545 method = PUT path = /v1/system/hardware/sleds/{sled_id}/provision-policy versions = all
27992025-04-30T06:43:50.909ZDEBGtest_utilization (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:53545 method = GET path = /v1/system/hardware/sleds-uninitialized versions = all
28002025-04-30T06:43:50.909ZDEBGtest_utilization (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:53545 method = GET path = /v1/system/hardware/switch-port versions = all
28012025-04-30T06:43:50.909ZDEBGtest_utilization (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:53545 method = GET path = /v1/system/hardware/switch-port/{port}/lldp/config versions = all
28022025-04-30T06:43:50.909ZDEBGtest_utilization (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:53545 method = POST path = /v1/system/hardware/switch-port/{port}/lldp/config versions = all
28032025-04-30T06:43:50.909ZDEBGtest_utilization (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:53545 method = DELETE path = /v1/system/hardware/switch-port/{port}/settings versions = all
28042025-04-30T06:43:50.909ZDEBGtest_utilization (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:53545 method = POST path = /v1/system/hardware/switch-port/{port}/settings versions = all
28052025-04-30T06:43:50.909ZDEBGtest_utilization (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:53545 method = GET path = /v1/system/hardware/switch-port/{port}/status versions = all
28062025-04-30T06:43:50.909ZDEBGtest_utilization (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:53545 method = GET path = /v1/system/hardware/switches versions = all
28072025-04-30T06:43:50.909ZDEBGtest_utilization (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:53545 method = GET path = /v1/system/hardware/switches/{switch_id} versions = all
28082025-04-30T06:43:50.909ZDEBGtest_utilization (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:53545 method = GET path = /v1/system/identity-providers versions = all
28092025-04-30T06:43:50.909ZDEBGtest_utilization (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:53545 method = POST path = /v1/system/identity-providers/local/users versions = all
28102025-04-30T06:43:50.909ZDEBGtest_utilization (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:53545 method = DELETE path = /v1/system/identity-providers/local/users/{user_id} versions = all
28112025-04-30T06:43:50.909ZDEBGtest_utilization (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:53545 method = POST path = /v1/system/identity-providers/local/users/{user_id}/set-password versions = all
28122025-04-30T06:43:50.909ZDEBGtest_utilization (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:53545 method = POST path = /v1/system/identity-providers/saml versions = all
28132025-04-30T06:43:50.909ZDEBGtest_utilization (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:53545 method = GET path = /v1/system/identity-providers/saml/{provider} versions = all
28142025-04-30T06:43:50.909ZDEBGtest_utilization (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:53545 method = GET path = /v1/system/ip-pools versions = all
28152025-04-30T06:43:50.909ZDEBGtest_utilization (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:53545 method = POST path = /v1/system/ip-pools versions = all
28162025-04-30T06:43:50.909ZDEBGtest_utilization (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:53545 method = DELETE path = /v1/system/ip-pools/{pool} versions = all
28172025-04-30T06:43:50.909ZDEBGtest_utilization (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:53545 method = GET path = /v1/system/ip-pools/{pool} versions = all
28182025-04-30T06:43:50.909ZDEBGtest_utilization (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:53545 method = PUT path = /v1/system/ip-pools/{pool} versions = all
28192025-04-30T06:43:50.909ZDEBGtest_utilization (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:53545 method = GET path = /v1/system/ip-pools/{pool}/ranges versions = all
28202025-04-30T06:43:50.909ZDEBGtest_utilization (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:53545 method = POST path = /v1/system/ip-pools/{pool}/ranges/add versions = all
28212025-04-30T06:43:50.909ZDEBGtest_utilization (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:53545 method = POST path = /v1/system/ip-pools/{pool}/ranges/remove versions = all
28222025-04-30T06:43:50.909ZDEBGtest_utilization (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:53545 method = GET path = /v1/system/ip-pools/{pool}/silos versions = all
28232025-04-30T06:43:50.909ZDEBGtest_utilization (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:53545 method = POST path = /v1/system/ip-pools/{pool}/silos versions = all
28242025-04-30T06:43:50.909ZDEBGtest_utilization (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:53545 method = DELETE path = /v1/system/ip-pools/{pool}/silos/{silo} versions = all
28252025-04-30T06:43:50.909ZDEBGtest_utilization (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:53545 method = PUT path = /v1/system/ip-pools/{pool}/silos/{silo} versions = all
28262025-04-30T06:43:50.909ZDEBGtest_utilization (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:53545 method = GET path = /v1/system/ip-pools/{pool}/utilization versions = all
28272025-04-30T06:43:50.909ZDEBGtest_utilization (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:53545 method = GET path = /v1/system/ip-pools-service versions = all
28282025-04-30T06:43:50.909ZDEBGtest_utilization (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:53545 method = GET path = /v1/system/ip-pools-service/ranges versions = all
28292025-04-30T06:43:50.909ZDEBGtest_utilization (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:53545 method = POST path = /v1/system/ip-pools-service/ranges/add versions = all
28302025-04-30T06:43:50.909ZDEBGtest_utilization (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:53545 method = POST path = /v1/system/ip-pools-service/ranges/remove versions = all
28312025-04-30T06:43:50.909ZDEBGtest_utilization (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:53545 method = GET path = /v1/system/metrics/{metric_name} versions = all
28322025-04-30T06:43:50.909ZDEBGtest_utilization (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:53545 method = GET path = /v1/system/networking/address-lot versions = all
28332025-04-30T06:43:50.910ZDEBGtest_utilization (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:53545 method = POST path = /v1/system/networking/address-lot versions = all
28342025-04-30T06:43:50.910ZDEBGtest_utilization (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:53545 method = DELETE path = /v1/system/networking/address-lot/{address_lot} versions = all
28352025-04-30T06:43:50.910ZDEBGtest_utilization (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:53545 method = GET path = /v1/system/networking/address-lot/{address_lot}/blocks versions = all
28362025-04-30T06:43:50.910ZDEBGtest_utilization (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:53545 method = GET path = /v1/system/networking/allow-list versions = all
28372025-04-30T06:43:50.910ZDEBGtest_utilization (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:53545 method = PUT path = /v1/system/networking/allow-list versions = all
28382025-04-30T06:43:50.910ZDEBGtest_utilization (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:53545 method = POST path = /v1/system/networking/bfd-disable versions = all
28392025-04-30T06:43:50.910ZDEBGtest_utilization (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:53545 method = POST path = /v1/system/networking/bfd-enable versions = all
28402025-04-30T06:43:50.910ZDEBGtest_utilization (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:53545 method = GET path = /v1/system/networking/bfd-status versions = all
28412025-04-30T06:43:50.910ZDEBGtest_utilization (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:53545 method = DELETE path = /v1/system/networking/bgp versions = all
28422025-04-30T06:43:50.910ZDEBGtest_utilization (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:53545 method = GET path = /v1/system/networking/bgp versions = all
28432025-04-30T06:43:50.910ZDEBGtest_utilization (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:53545 method = POST path = /v1/system/networking/bgp versions = all
28442025-04-30T06:43:50.910ZDEBGtest_utilization (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:53545 method = GET path = /v1/system/networking/bgp-announce-set versions = all
28452025-04-30T06:43:50.910ZDEBGtest_utilization (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:53545 method = PUT path = /v1/system/networking/bgp-announce-set versions = all
28462025-04-30T06:43:50.910ZDEBGtest_utilization (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:53545 method = DELETE path = /v1/system/networking/bgp-announce-set/{announce_set} versions = all
28472025-04-30T06:43:50.910ZDEBGtest_utilization (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:53545 method = GET path = /v1/system/networking/bgp-announce-set/{announce_set}/announcement versions = all
28482025-04-30T06:43:50.910ZDEBGtest_utilization (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:53545 method = GET path = /v1/system/networking/bgp-exported versions = all
28492025-04-30T06:43:50.910ZDEBGtest_utilization (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:53545 method = GET path = /v1/system/networking/bgp-message-history versions = all
28502025-04-30T06:43:50.910ZDEBGtest_utilization (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:53545 method = GET path = /v1/system/networking/bgp-routes-ipv4 versions = all
28512025-04-30T06:43:50.910ZDEBGtest_utilization (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:53545 method = GET path = /v1/system/networking/bgp-status versions = all
28522025-04-30T06:43:50.910ZDEBGtest_utilization (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:53545 method = GET path = /v1/system/networking/loopback-address versions = all
28532025-04-30T06:43:50.910ZDEBGtest_utilization (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:53545 method = POST path = /v1/system/networking/loopback-address versions = all
28542025-04-30T06:43:50.910ZDEBGtest_utilization (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:53545 method = DELETE path = /v1/system/networking/loopback-address/{rack_id}/{switch_location}/{address}/{subnet_mask} versions = all
28552025-04-30T06:43:50.910ZDEBGtest_utilization (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:53545 method = DELETE path = /v1/system/networking/switch-port-settings versions = all
28562025-04-30T06:43:50.910ZDEBGtest_utilization (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:53545 method = GET path = /v1/system/networking/switch-port-settings versions = all
28572025-04-30T06:43:50.910ZDEBGtest_utilization (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:53545 method = POST path = /v1/system/networking/switch-port-settings versions = all
28582025-04-30T06:43:50.910ZDEBGtest_utilization (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:53545 method = GET path = /v1/system/networking/switch-port-settings/{port} versions = all
28592025-04-30T06:43:50.910ZDEBGtest_utilization (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:53545 method = GET path = /v1/system/policy versions = all
28602025-04-30T06:43:50.910ZDEBGtest_utilization (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:53545 method = PUT path = /v1/system/policy versions = all
28612025-04-30T06:43:50.910ZDEBGtest_utilization (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:53545 method = GET path = /v1/system/roles versions = all
28622025-04-30T06:43:50.910ZDEBGtest_utilization (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:53545 method = GET path = /v1/system/roles/{role_name} versions = all
28632025-04-30T06:43:50.910ZDEBGtest_utilization (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:53545 method = GET path = /v1/system/silo-quotas versions = all
28642025-04-30T06:43:50.910ZDEBGtest_utilization (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:53545 method = GET path = /v1/system/silos versions = all
28652025-04-30T06:43:50.910ZDEBGtest_utilization (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:53545 method = POST path = /v1/system/silos versions = all
28662025-04-30T06:43:50.910ZDEBGtest_utilization (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:53545 method = DELETE path = /v1/system/silos/{silo} versions = all
28672025-04-30T06:43:50.910ZDEBGtest_utilization (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:53545 method = GET path = /v1/system/silos/{silo} versions = all
28682025-04-30T06:43:50.910ZDEBGtest_utilization (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:53545 method = GET path = /v1/system/silos/{silo}/ip-pools versions = all
28692025-04-30T06:43:50.910ZDEBGtest_utilization (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:53545 method = GET path = /v1/system/silos/{silo}/policy versions = all
28702025-04-30T06:43:50.910ZDEBGtest_utilization (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:53545 method = PUT path = /v1/system/silos/{silo}/policy versions = all
28712025-04-30T06:43:50.910ZDEBGtest_utilization (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:53545 method = GET path = /v1/system/silos/{silo}/quotas versions = all
28722025-04-30T06:43:50.910ZDEBGtest_utilization (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:53545 method = PUT path = /v1/system/silos/{silo}/quotas versions = all
28732025-04-30T06:43:50.910ZDEBGtest_utilization (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:53545 method = POST path = /v1/system/timeseries/query versions = all
28742025-04-30T06:43:50.910ZDEBGtest_utilization (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:53545 method = GET path = /v1/system/timeseries/schemas versions = all
28752025-04-30T06:43:50.911ZDEBGtest_utilization (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:53545 method = PUT path = /v1/system/update/repository versions = all
28762025-04-30T06:43:50.911ZDEBGtest_utilization (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:53545 method = GET path = /v1/system/update/repository/{system_version} versions = all
28772025-04-30T06:43:50.911ZDEBGtest_utilization (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:53545 method = GET path = /v1/system/update/target-release versions = all
28782025-04-30T06:43:50.911ZDEBGtest_utilization (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:53545 method = PUT path = /v1/system/update/target-release versions = all
28792025-04-30T06:43:50.911ZDEBGtest_utilization (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:53545 method = GET path = /v1/system/users versions = all
28802025-04-30T06:43:50.911ZDEBGtest_utilization (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:53545 method = GET path = /v1/system/users/{user_id} versions = all
28812025-04-30T06:43:50.911ZDEBGtest_utilization (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:53545 method = GET path = /v1/system/users-builtin versions = all
28822025-04-30T06:43:50.911ZDEBGtest_utilization (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:53545 method = GET path = /v1/system/users-builtin/{user} versions = all
28832025-04-30T06:43:50.911ZDEBGtest_utilization (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:53545 method = GET path = /v1/system/utilization/silos versions = all
28842025-04-30T06:43:50.911ZDEBGtest_utilization (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:53545 method = GET path = /v1/system/utilization/silos/{silo} versions = all
28852025-04-30T06:43:50.911ZDEBGtest_utilization (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:53545 method = POST path = /v1/timeseries/query versions = all
28862025-04-30T06:43:50.911ZDEBGtest_utilization (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:53545 method = GET path = /v1/users versions = all
28872025-04-30T06:43:50.911ZDEBGtest_utilization (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:53545 method = GET path = /v1/utilization versions = all
28882025-04-30T06:43:50.911ZDEBGtest_utilization (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:53545 method = GET path = /v1/vpc-firewall-rules versions = all
28892025-04-30T06:43:50.911ZDEBGtest_utilization (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:53545 method = PUT path = /v1/vpc-firewall-rules versions = all
28902025-04-30T06:43:50.911ZDEBGtest_utilization (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:53545 method = GET path = /v1/vpc-router-routes versions = all
28912025-04-30T06:43:50.911ZDEBGtest_utilization (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:53545 method = POST path = /v1/vpc-router-routes versions = all
28922025-04-30T06:43:50.911ZDEBGtest_utilization (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:53545 method = DELETE path = /v1/vpc-router-routes/{route} versions = all
28932025-04-30T06:43:50.911ZDEBGtest_utilization (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:53545 method = GET path = /v1/vpc-router-routes/{route} versions = all
28942025-04-30T06:43:50.911ZDEBGtest_utilization (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:53545 method = PUT path = /v1/vpc-router-routes/{route} versions = all
28952025-04-30T06:43:50.911ZDEBGtest_utilization (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:53545 method = GET path = /v1/vpc-routers versions = all
28962025-04-30T06:43:50.911ZDEBGtest_utilization (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:53545 method = POST path = /v1/vpc-routers versions = all
28972025-04-30T06:43:50.911ZDEBGtest_utilization (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:53545 method = DELETE path = /v1/vpc-routers/{router} versions = all
28982025-04-30T06:43:50.911ZDEBGtest_utilization (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:53545 method = GET path = /v1/vpc-routers/{router} versions = all
28992025-04-30T06:43:50.911ZDEBGtest_utilization (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:53545 method = PUT path = /v1/vpc-routers/{router} versions = all
29002025-04-30T06:43:50.911ZDEBGtest_utilization (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:53545 method = GET path = /v1/vpc-subnets versions = all
29012025-04-30T06:43:50.911ZDEBGtest_utilization (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:53545 method = POST path = /v1/vpc-subnets versions = all
29022025-04-30T06:43:50.911ZDEBGtest_utilization (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:53545 method = DELETE path = /v1/vpc-subnets/{subnet} versions = all
29032025-04-30T06:43:50.911ZDEBGtest_utilization (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:53545 method = GET path = /v1/vpc-subnets/{subnet} versions = all
29042025-04-30T06:43:50.911ZDEBGtest_utilization (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:53545 method = PUT path = /v1/vpc-subnets/{subnet} versions = all
29052025-04-30T06:43:50.911ZDEBGtest_utilization (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:53545 method = GET path = /v1/vpc-subnets/{subnet}/network-interfaces versions = all
29062025-04-30T06:43:50.911ZDEBGtest_utilization (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:53545 method = GET path = /v1/vpcs versions = all
29072025-04-30T06:43:50.911ZDEBGtest_utilization (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:53545 method = POST path = /v1/vpcs versions = all
29082025-04-30T06:43:50.911ZDEBGtest_utilization (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:53545 method = DELETE path = /v1/vpcs/{vpc} versions = all
29092025-04-30T06:43:50.911ZDEBGtest_utilization (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:53545 method = GET path = /v1/vpcs/{vpc} versions = all
29102025-04-30T06:43:50.911ZDEBGtest_utilization (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:53545 method = PUT path = /v1/vpcs/{vpc} versions = all
29112025-04-30T06:43:50.911ZDEBGtest_utilization (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:53545 method = GET path = /v1/webhooks/deliveries versions = all
29122025-04-30T06:43:50.911ZDEBGtest_utilization (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:53545 method = POST path = /v1/webhooks/deliveries/{event_id}/resend versions = all
29132025-04-30T06:43:50.911ZDEBGtest_utilization (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:53545 method = GET path = /v1/webhooks/event-classes versions = all
29142025-04-30T06:43:50.911ZDEBGtest_utilization (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:53545 method = GET path = /v1/webhooks/receivers versions = all
29152025-04-30T06:43:50.911ZDEBGtest_utilization (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:53545 method = POST path = /v1/webhooks/receivers versions = all
29162025-04-30T06:43:50.911ZDEBGtest_utilization (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:53545 method = DELETE path = /v1/webhooks/receivers/{receiver} versions = all
29172025-04-30T06:43:50.911ZDEBGtest_utilization (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:53545 method = GET path = /v1/webhooks/receivers/{receiver} versions = all
29182025-04-30T06:43:50.912ZDEBGtest_utilization (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:53545 method = PUT path = /v1/webhooks/receivers/{receiver} versions = all
29192025-04-30T06:43:50.912ZDEBGtest_utilization (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:53545 method = POST path = /v1/webhooks/receivers/{receiver}/probe versions = all
29202025-04-30T06:43:50.912ZDEBGtest_utilization (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:53545 method = POST path = /v1/webhooks/receivers/{receiver}/subscriptions versions = all
29212025-04-30T06:43:50.912ZDEBGtest_utilization (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:53545 method = DELETE path = /v1/webhooks/receivers/{receiver}/subscriptions/{subscription} versions = all
29222025-04-30T06:43:50.912ZDEBGtest_utilization (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:53545 method = GET path = /v1/webhooks/secrets versions = all
29232025-04-30T06:43:50.912ZDEBGtest_utilization (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:53545 method = POST path = /v1/webhooks/secrets versions = all
29242025-04-30T06:43:50.912ZDEBGtest_utilization (dropshot_external_techport): registered endpoint local_addr = 127.0.0.1:53545 method = DELETE path = /v1/webhooks/secrets/{secret_id} versions = all
29252025-04-30T06:43:50.912ZINFOtest_utilization (dropshot_external_techport): listening local_addr = 127.0.0.1:53545
29262025-04-30T06:43:50.912ZDEBGtest_utilization (dropshot_external_techport): successfully registered DTrace USDT probes local_addr = 127.0.0.1:53545
29272025-04-30T06:43:50.912ZDEBGtest_utilization (nexus-producer-server): registered DTrace probes
29282025-04-30T06:43:50.912ZDEBGtest_utilization (dropshot): registered endpoint local_addr = 127.0.0.1:57688 method = GET path = /{producer_id} versions = all
29292025-04-30T06:43:50.912ZINFOtest_utilization (dropshot): listening file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:328 local_addr = 127.0.0.1:57688
29302025-04-30T06:43:50.912ZDEBGtest_utilization (dropshot): successfully registered DTrace USDT probes local_addr = 127.0.0.1:57688
29312025-04-30T06:43:50.912ZDEBGtest_utilization (nexus-producer-server): Requested any available port, Dropshot server has been bound to 127.0.0.1:57688
29322025-04-30T06:43:50.912ZDEBGtest_utilization (nexus-producer-server): Nexus IP provided explicitly, will use it addr = 127.0.0.1:32956
29332025-04-30T06:43:50.912ZDEBGtest_utilization (nexus-producer-server): starting producer registration task
29342025-04-30T06:43:50.912ZINFOtest_utilization (nexus-producer-server): starting oximeter metric producer server address = 127.0.0.1:57688 file = oximeter/producer/src/lib.rs:279 interval = 10s producer_id = ac55d8d9-a298-4534-898b-beee1c9e4e52
29352025-04-30T06:43:50.912ZDEBGtest_utilization (init_with_steps): Running step start_oximeter
29362025-04-30T06:43:50.912ZDEBGtest_utilization: Starting Oximeter
29372025-04-30T06:43:50.912ZDEBGtest_utilization (oximeter): registered DTrace probes
29382025-04-30T06:43:50.912ZINFOtest_utilization (oximeter): starting oximeter server file = oximeter/collector/src/lib.rs:236
29392025-04-30T06:43:50.912ZDEBGtest_utilization (oximeter): creating ClickHouse client
29402025-04-30T06:43:50.912ZDEBGtest_utilization (clickhouse-client): registered USDT probes collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 4d1485f0-110e-4161-9a33-c4610d68be5d
29412025-04-30T06:43:50.913ZDEBGtest_utilization (producer-registration-task): registering as oximeter producer with Nexus registration_count = 0
29422025-04-30T06:43:50.913ZDEBGtest_utilization (producer-registration-task): will register with Nexus at 127.0.0.1:32956
29432025-04-30T06:43:50.978ZDEBGtest_utilization (producer-registration-task): client request body = Some(Body) method = POST uri = http://127.0.0.1:32956/metrics/producers
29442025-04-30T06:43:50.980ZINFOtest_utilization (dropshot_internal): accepted connection local_addr = 127.0.0.1:32956 remote_addr = 127.0.0.1:63420
29452025-04-30T06:43:50.982ZTRACtest_utilization (dropshot_internal): incoming request local_addr = 127.0.0.1:32956 method = POST remote_addr = 127.0.0.1:63420 req_id = c22e8590-4406-4bfe-a112-4056764c13b7 uri = /metrics/producers
29462025-04-30T06:43:50.983ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "1729ecfa-b778-4003-812e-c9ecd18d6df4", "content-length": "729", "date": "Wed, 30 Apr 2025 06:43:50 GMT"} })
29472025-04-30T06:43:50.983ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0/component/rot/cfpa
29482025-04-30T06:43:50.986ZTRACtest_utilization (dropshot_internal): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = 127.0.0.1:32956 method = POST remote_addr = 127.0.0.1:63420 req_id = c22e8590-4406-4bfe-a112-4056764c13b7 resource = Database uri = /metrics/producers
29492025-04-30T06:43:50.986ZDEBGtest_utilization (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = 127.0.0.1:32956 method = POST remote_addr = 127.0.0.1:63420 req_id = c22e8590-4406-4bfe-a112-4056764c13b7 roles = RoleSet { roles: {} } uri = /metrics/producers
29502025-04-30T06:43:50.990ZDEBGtest_utilization (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = 127.0.0.1:32956 method = POST remote_addr = 127.0.0.1:63420 req_id = c22e8590-4406-4bfe-a112-4056764c13b7 resource = Database result = Ok(()) uri = /metrics/producers
29512025-04-30T06:43:50.991ZTRACtest_utilization (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 4d1485f0-110e-4161-9a33-c4610d68be5d sql = SELECT MAX(value) AS max_version FROM oximeter.version;
29522025-04-30T06:43:50.998ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "a1ebac13-81a0-41a6-b82a-ed3fe4a84248", "content-length": "728", "date": "Wed, 30 Apr 2025 06:43:50 GMT"} })
29532025-04-30T06:43:50.999ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1
29542025-04-30T06:43:51.001ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1", status: 200, headers: {"content-type": "application/json", "x-request-id": "5c8353ec-59f9-4569-b113-8c98d6d370b3", "content-length": "723", "date": "Wed, 30 Apr 2025 06:43:50 GMT"} })
29552025-04-30T06:43:51.001ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1/component/sp/caboose?firmware_slot=0
29562025-04-30T06:43:51.005ZWARNtest_utilization (clickhouse-client): oximeter database does not exist, or is out-of-date collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 file = oximeter/db/src/client/mod.rs:894 id = 4d1485f0-110e-4161-9a33-c4610d68be5d
29572025-04-30T06:43:51.005ZDEBGtest_utilization (oximeter-agent): oximeter database does not exist, creating collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
29582025-04-30T06:43:51.005ZINFOtest_utilization (clickhouse-client): reading db version collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 file = oximeter/db/src/client/mod.rs:794 id = 4d1485f0-110e-4161-9a33-c4610d68be5d
29592025-04-30T06:43:51.006ZTRACtest_utilization (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 4d1485f0-110e-4161-9a33-c4610d68be5d sql = SELECT MAX(value) AS max_version FROM oximeter.version;
29602025-04-30T06:43:51.007ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "4d10b8f1-7df6-4740-abfe-3af69b6198d4", "content-length": "109", "date": "Wed, 30 Apr 2025 06:43:50 GMT"} })
29612025-04-30T06:43:51.007ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1/component/sp/caboose?firmware_slot=1
29622025-04-30T06:43:51.007ZWARNtest_utilization (clickhouse-client): oximeter database does not exist, or is out-of-date collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 file = oximeter/db/src/client/mod.rs:894 id = 4d1485f0-110e-4161-9a33-c4610d68be5d
29632025-04-30T06:43:51.007ZINFOtest_utilization (clickhouse-client): read oximeter database version collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 file = oximeter/db/src/client/mod.rs:799 id = 4d1485f0-110e-4161-9a33-c4610d68be5d version = 0
29642025-04-30T06:43:51.007ZINFOtest_utilization (clickhouse-client): initializing oximeter schema collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 file = oximeter/db/src/client/mod.rs:804 id = 4d1485f0-110e-4161-9a33-c4610d68be5d
29652025-04-30T06:43:51.007ZDEBGtest_utilization (clickhouse-client): initializing ClickHouse database collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 4d1485f0-110e-4161-9a33-c4610d68be5d
29662025-04-30T06:43:51.008ZTRACtest_utilization (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 4d1485f0-110e-4161-9a33-c4610d68be5d sql = CREATE DATABASE IF NOT EXISTS oximeter
29672025-04-30T06:43:51.010ZTRACtest_utilization (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 4d1485f0-110e-4161-9a33-c4610d68be5d sql = \n\n/* The version table contains metadata about the `oximeter` database */\nCREATE TABLE IF NOT EXISTS oximeter.version\n(\n value UInt64,\n timestamp DateTime64(9, 'UTC')\n)\nENGINE = MergeTree()\nORDER BY (value, timestamp)
29682025-04-30T06:43:51.015ZTRACtest_utilization (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 4d1485f0-110e-4161-9a33-c4610d68be5d sql = \n\n/* The measurement tables contain all individual samples from each timeseries.\n *\n * Each table stores a single datum type, and otherwise contains nearly the same\n * structure. The primary sorting key is on the timeseries name, key, and then\n * timestamp, so that all timeseries from the same schema are grouped, followed\n * by all samples from the same timeseries.\n *\n * This reflects that one usually looks up the _key_ in one or more field table,\n * and then uses that to index quickly into the measurements tables.\n */\nCREATE TABLE IF NOT EXISTS oximeter.measurements_bool\n(\n timeseries_name String,\n timeseries_key UInt64,\n timestamp DateTime64(9, 'UTC'),\n datum Nullable(Bool)\n)\nENGINE = MergeTree()\nORDER BY (timeseries_name, timeseries_key, timestamp)\nTTL toDateTime(timestamp) + INTERVAL 30 DAY
29692025-04-30T06:43:51.016ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "929b90ca-9fe5-4797-ba0b-5a5efef2e64d", "content-length": "109", "date": "Wed, 30 Apr 2025 06:43:50 GMT"} })
29702025-04-30T06:43:51.016ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1/component/rot/caboose?firmware_slot=0
29712025-04-30T06:43:51.020ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "61772233-0503-4854-a73e-477403b80f41", "content-length": "169", "date": "Wed, 30 Apr 2025 06:43:50 GMT"} })
29722025-04-30T06:43:51.020ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1/component/rot/caboose?firmware_slot=1
29732025-04-30T06:43:51.020ZTRACtest_utilization (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 4d1485f0-110e-4161-9a33-c4610d68be5d sql = \n\nCREATE TABLE IF NOT EXISTS oximeter.measurements_i8\n(\n timeseries_name String,\n timeseries_key UInt64,\n timestamp DateTime64(9, 'UTC'),\n datum Nullable(Int8)\n)\nENGINE = MergeTree()\nORDER BY (timeseries_name, timeseries_key, timestamp)\nTTL toDateTime(timestamp) + INTERVAL 30 DAY
29742025-04-30T06:43:51.024ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "76bb613e-f175-4bd5-9289-b94e77fe5469", "content-length": "169", "date": "Wed, 30 Apr 2025 06:43:50 GMT"} })
29752025-04-30T06:43:51.024ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1/component/stage0/caboose?firmware_slot=0
29762025-04-30T06:43:51.025ZTRACtest_utilization (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 4d1485f0-110e-4161-9a33-c4610d68be5d sql = \n\nCREATE TABLE IF NOT EXISTS oximeter.measurements_u8\n(\n timeseries_name String,\n timeseries_key UInt64,\n timestamp DateTime64(9, 'UTC'),\n datum Nullable(UInt8)\n)\nENGINE = MergeTree()\nORDER BY (timeseries_name, timeseries_key, timestamp)\nTTL toDateTime(timestamp) + INTERVAL 30 DAY
29772025-04-30T06:43:51.027ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "bb425856-85b9-4207-9251-0a78fc072f9b", "content-length": "178", "date": "Wed, 30 Apr 2025 06:43:50 GMT"} })
29782025-04-30T06:43:51.027ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1/component/stage0/caboose?firmware_slot=1
29792025-04-30T06:43:51.029ZTRACtest_utilization (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 4d1485f0-110e-4161-9a33-c4610d68be5d sql = \n\nCREATE TABLE IF NOT EXISTS oximeter.measurements_i16\n(\n timeseries_name String,\n timeseries_key UInt64,\n timestamp DateTime64(9, 'UTC'),\n datum Nullable(Int16)\n)\nENGINE = MergeTree()\nORDER BY (timeseries_name, timeseries_key, timestamp)\nTTL toDateTime(timestamp) + INTERVAL 30 DAY
29802025-04-30T06:43:51.031ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "c3fb7eff-5ed9-4dc4-894a-d24c381ebe00", "content-length": "178", "date": "Wed, 30 Apr 2025 06:43:50 GMT"} })
29812025-04-30T06:43:51.031ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1/component/rot/cmpa
29822025-04-30T06:43:51.032ZTRACtest_utilization (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 4d1485f0-110e-4161-9a33-c4610d68be5d sql = \n\nCREATE TABLE IF NOT EXISTS oximeter.measurements_u16\n(\n timeseries_name String,\n timeseries_key UInt64,\n timestamp DateTime64(9, 'UTC'),\n datum Nullable(UInt16)\n)\nENGINE = MergeTree()\nORDER BY (timeseries_name, timeseries_key, timestamp)\nTTL toDateTime(timestamp) + INTERVAL 30 DAY
29832025-04-30T06:43:51.033ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "d24f907c-36ab-422f-a704-be6880d6a5af", "content-length": "702", "date": "Wed, 30 Apr 2025 06:43:50 GMT"} })
29842025-04-30T06:43:51.033ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1/component/rot/cfpa
29852025-04-30T06:43:51.034ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "d0de7c67-820b-49e7-b3b4-3bef345a5cb7", "content-length": "727", "date": "Wed, 30 Apr 2025 06:43:50 GMT"} })
29862025-04-30T06:43:51.035ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1/component/rot/cfpa
29872025-04-30T06:43:51.036ZTRACtest_utilization (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 4d1485f0-110e-4161-9a33-c4610d68be5d sql = \n\nCREATE TABLE IF NOT EXISTS oximeter.measurements_i32\n(\n timeseries_name String,\n timeseries_key UInt64,\n timestamp DateTime64(9, 'UTC'),\n datum Nullable(Int32)\n)\nENGINE = MergeTree()\nORDER BY (timeseries_name, timeseries_key, timestamp)\nTTL toDateTime(timestamp) + INTERVAL 30 DAY
29882025-04-30T06:43:51.037ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "f9bcb33b-4d60-40e4-b3b9-1ab380f896a4", "content-length": "729", "date": "Wed, 30 Apr 2025 06:43:50 GMT"} })
29892025-04-30T06:43:51.038ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1/component/rot/cfpa
29902025-04-30T06:43:51.039ZTRACtest_utilization (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 4d1485f0-110e-4161-9a33-c4610d68be5d sql = \n\nCREATE TABLE IF NOT EXISTS oximeter.measurements_u32\n(\n timeseries_name String,\n timeseries_key UInt64,\n timestamp DateTime64(9, 'UTC'),\n datum Nullable(UInt32)\n)\nENGINE = MergeTree()\nORDER BY (timeseries_name, timeseries_key, timestamp)\nTTL toDateTime(timestamp) + INTERVAL 30 DAY
29912025-04-30T06:43:51.039ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "34fa5f3a-d0b7-4684-a690-5e8dccf73010", "content-length": "728", "date": "Wed, 30 Apr 2025 06:43:50 GMT"} })
29922025-04-30T06:43:51.039ZTRACtest_utilization (BackgroundTasks): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet
29932025-04-30T06:43:51.039ZTRACtest_utilization (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
29942025-04-30T06:43:51.039ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
29952025-04-30T06:43:51.040ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
29962025-04-30T06:43:51.040ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
29972025-04-30T06:43:51.042ZTRACtest_utilization (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 4d1485f0-110e-4161-9a33-c4610d68be5d sql = \n\nCREATE TABLE IF NOT EXISTS oximeter.measurements_i64\n(\n timeseries_name String,\n timeseries_key UInt64,\n timestamp DateTime64(9, 'UTC'),\n datum Nullable(Int64)\n)\nENGINE = MergeTree()\nORDER BY (timeseries_name, timeseries_key, timestamp)\nTTL toDateTime(timestamp) + INTERVAL 30 DAY
29982025-04-30T06:43:51.043ZINFOtest_utilization (dropshot_internal): request completed error_message_external = Service Unavailable error_message_internal = no Oximeter instances available for assignment latency_us = 61057 local_addr = 127.0.0.1:32956 method = POST remote_addr = 127.0.0.1:63420 req_id = c22e8590-4406-4bfe-a112-4056764c13b7 response_code = 503 uri = /metrics/producers
29992025-04-30T06:43:51.044ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
30002025-04-30T06:43:51.049ZDEBGtest_utilization (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet result = Ok(())
30012025-04-30T06:43:51.049ZTRACtest_utilization (BackgroundTasks): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet
30022025-04-30T06:43:51.049ZTRACtest_utilization (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
30032025-04-30T06:43:51.052ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
30042025-04-30T06:43:51.052ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
30052025-04-30T06:43:51.052ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
30062025-04-30T06:43:51.061ZTRACtest_utilization (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 4d1485f0-110e-4161-9a33-c4610d68be5d sql = \n\nCREATE TABLE IF NOT EXISTS oximeter.measurements_u64\n(\n timeseries_name String,\n timeseries_key UInt64,\n timestamp DateTime64(9, 'UTC'),\n datum Nullable(UInt64)\n)\nENGINE = MergeTree()\nORDER BY (timeseries_name, timeseries_key, timestamp)\nTTL toDateTime(timestamp) + INTERVAL 30 DAY
30072025-04-30T06:43:51.062ZDEBGtest_utilization (producer-registration-task): client response result = Ok(Response { url: "http://127.0.0.1:32956/metrics/producers", status: 503, headers: {"content-type": "application/json", "x-request-id": "c22e8590-4406-4bfe-a112-4056764c13b7", "content-length": "133", "date": "Wed, 30 Apr 2025 06:43:50 GMT"} })
30082025-04-30T06:43:51.068ZWARNtest_utilization (producer-registration-task): failed to register with Nexus, will retry count = 0 delay = 155.321689ms error = "Error Response: status: 503 Service Unavailable; headers: {\\"content-type\\": \\"application/json\\", \\"x-request-id\\": \\"c22e8590-4406-4bfe-a112-4056764c13b7\\", \\"content-length\\": \\"133\\", \\"date\\": \\"Wed, 30 Apr 2025 06:43:50 GMT\\"}; value: Error { error_code: Some(\\"ServiceNotAvailable\\"), message: \\"Service Unavailable\\", request_id: \\"c22e8590-4406-4bfe-a112-4056764c13b7\\" }" file = oximeter/producer/src/lib.rs:371
30092025-04-30T06:43:51.069ZTRACtest_utilization (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 4d1485f0-110e-4161-9a33-c4610d68be5d sql = \n\nCREATE TABLE IF NOT EXISTS oximeter.measurements_f32\n(\n timeseries_name String,\n timeseries_key UInt64,\n timestamp DateTime64(9, 'UTC'),\n datum Nullable(Float32)\n)\nENGINE = MergeTree()\nORDER BY (timeseries_name, timeseries_key, timestamp)\nTTL toDateTime(timestamp) + INTERVAL 30 DAY
30102025-04-30T06:43:51.072ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
30112025-04-30T06:43:51.077ZDEBGtest_utilization (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet result = Ok(())
30122025-04-30T06:43:51.077ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
30132025-04-30T06:43:51.077ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
30142025-04-30T06:43:51.077ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
30152025-04-30T06:43:51.078ZTRACtest_utilization (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 4d1485f0-110e-4161-9a33-c4610d68be5d sql = \n\nCREATE TABLE IF NOT EXISTS oximeter.measurements_f64\n(\n timeseries_name String,\n timeseries_key UInt64,\n timestamp DateTime64(9, 'UTC'),\n datum Nullable(Float64)\n)\nENGINE = MergeTree()\nORDER BY (timeseries_name, timeseries_key, timestamp)\nTTL toDateTime(timestamp) + INTERVAL 30 DAY
30162025-04-30T06:43:51.082ZTRACtest_utilization (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 4d1485f0-110e-4161-9a33-c4610d68be5d sql = \n\nCREATE TABLE IF NOT EXISTS oximeter.measurements_string\n(\n timeseries_name String,\n timeseries_key UInt64,\n timestamp DateTime64(9, 'UTC'),\n datum Nullable(String)\n)\nENGINE = MergeTree()\nORDER BY (timeseries_name, timeseries_key, timestamp)\nTTL toDateTime(timestamp) + INTERVAL 30 DAY
30172025-04-30T06:43:51.133ZDEBGtest_utilization (BackgroundTasks): begin collection from Sled Agent background_task = inventory_collection sled_agent_url = http://[::1]:39365
30182025-04-30T06:43:51.133ZDEBGtest_utilization (BackgroundTasks): client request SledAgent = http://[::1]:39365 background_task = inventory_collection body = None method = GET uri = http://[::1]:39365/inventory
30192025-04-30T06:43:51.134ZTRACtest_utilization (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 4d1485f0-110e-4161-9a33-c4610d68be5d sql = \n\nCREATE TABLE IF NOT EXISTS oximeter.measurements_bytes\n(\n timeseries_name String,\n timeseries_key UInt64,\n timestamp DateTime64(9, 'UTC'),\n /*\n * NOTE: Right now we can't unambiguously record a nullable byte array.\n * Arrays cannot be nested in `Nullable()` types, and encoding the array as\n * a string isn't palatable for a few reasons.\n * See: https://github.com/oxidecomputer/omicron/issues/4551 for more\n * details.\n */\n datum Array(UInt8)\n)\nENGINE = MergeTree()\nORDER BY (timeseries_name, timeseries_key, timestamp)\nTTL toDateTime(timestamp) + INTERVAL 30 DAY
30202025-04-30T06:43:51.134ZINFOtest_utilization (dropshot): accepted connection local_addr = [::1]:39365 remote_addr = [::1]:64992 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
30212025-04-30T06:43:51.134ZTRACtest_utilization (dropshot): incoming request local_addr = [::1]:39365 method = GET remote_addr = [::1]:64992 req_id = 7d7effd7-70e1-4650-a3f4-f00960df5de2 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /inventory
30222025-04-30T06:43:51.135ZINFOtest_utilization (dropshot): request completed latency_us = 593 local_addr = [::1]:39365 method = GET remote_addr = [::1]:64992 req_id = 7d7effd7-70e1-4650-a3f4-f00960df5de2 response_code = 200 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /inventory
30232025-04-30T06:43:51.135ZDEBGtest_utilization (BackgroundTasks): client response SledAgent = http://[::1]:39365 background_task = inventory_collection result = Ok(Response { url: "http://[::1]:39365/inventory", status: 200, headers: {"content-type": "application/json", "x-request-id": "7d7effd7-70e1-4650-a3f4-f00960df5de2", "content-length": "2769", "date": "Wed, 30 Apr 2025 06:43:50 GMT"} })
30242025-04-30T06:43:51.139ZDEBGtest_utilization (BackgroundTasks): begin collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
30252025-04-30T06:43:51.139ZDEBGtest_utilization (BackgroundTasks): end collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
30262025-04-30T06:43:51.139ZDEBGtest_utilization (BackgroundTasks): finished collection background_task = inventory_collection
30272025-04-30T06:43:51.141ZTRACtest_utilization (BackgroundTasks): authorize begin action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory
30282025-04-30T06:43:51.141ZTRACtest_utilization (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
30292025-04-30T06:43:51.141ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
30302025-04-30T06:43:51.141ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
30312025-04-30T06:43:51.142ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
30322025-04-30T06:43:51.143ZTRACtest_utilization (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 4d1485f0-110e-4161-9a33-c4610d68be5d sql = \n\nCREATE TABLE IF NOT EXISTS oximeter.measurements_cumulativei64\n(\n timeseries_name String,\n timeseries_key UInt64,\n start_time DateTime64(9, 'UTC'),\n timestamp DateTime64(9, 'UTC'),\n datum Nullable(Int64)\n)\nENGINE = MergeTree()\nORDER BY (timeseries_name, timeseries_key, start_time, timestamp)\nTTL toDateTime(timestamp) + INTERVAL 30 DAY
30332025-04-30T06:43:51.145ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
30342025-04-30T06:43:51.149ZDEBGtest_utilization (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory result = Ok(())
30352025-04-30T06:43:51.149ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
30362025-04-30T06:43:51.149ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
30372025-04-30T06:43:51.150ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
30382025-04-30T06:43:51.151ZTRACtest_utilization (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 4d1485f0-110e-4161-9a33-c4610d68be5d sql = \n\nCREATE TABLE IF NOT EXISTS oximeter.measurements_cumulativeu64\n(\n timeseries_name String,\n timeseries_key UInt64,\n start_time DateTime64(9, 'UTC'),\n timestamp DateTime64(9, 'UTC'),\n datum Nullable(UInt64)\n)\nENGINE = MergeTree()\nORDER BY (timeseries_name, timeseries_key, start_time, timestamp)\nTTL toDateTime(timestamp) + INTERVAL 30 DAY
30392025-04-30T06:43:51.162ZTRACtest_utilization (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 4d1485f0-110e-4161-9a33-c4610d68be5d sql = \n\nCREATE TABLE IF NOT EXISTS oximeter.measurements_cumulativef32\n(\n timeseries_name String,\n timeseries_key UInt64,\n start_time DateTime64(9, 'UTC'),\n timestamp DateTime64(9, 'UTC'),\n datum Nullable(Float32)\n)\nENGINE = MergeTree()\nORDER BY (timeseries_name, timeseries_key, start_time, timestamp)\nTTL toDateTime(timestamp) + INTERVAL 30 DAY
30402025-04-30T06:43:51.165ZTRACtest_utilization (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 4d1485f0-110e-4161-9a33-c4610d68be5d sql = \n\n\nCREATE TABLE IF NOT EXISTS oximeter.measurements_cumulativef64\n(\n timeseries_name String,\n timeseries_key UInt64,\n start_time DateTime64(9, 'UTC'),\n timestamp DateTime64(9, 'UTC'),\n datum Nullable(Float64)\n)\nENGINE = MergeTree()\nORDER BY (timeseries_name, timeseries_key, start_time, timestamp)\nTTL toDateTime(timestamp) + INTERVAL 30 DAY
30412025-04-30T06:43:51.172ZTRACtest_utilization (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 4d1485f0-110e-4161-9a33-c4610d68be5d sql = \n\nCREATE TABLE IF NOT EXISTS oximeter.measurements_histogrami8\n(\n timeseries_name String,\n timeseries_key UInt64,\n start_time DateTime64(9, 'UTC'),\n timestamp DateTime64(9, 'UTC'),\n /*\n * NOTE: Array types cannot be Nullable, see\n * https://clickhouse.com/docs/en/sql-reference/data-types/nullable\n * for more details.\n *\n * This means we need to use empty arrays to indicate a missing value. This\n * is unfortunate, and at this point relies on the fact that an\n * `oximeter::Histogram` cannot have zero bins. If that changes, we'll need\n * to figure out another way to represent missing samples here.\n */\n bins Array(Int8),\n counts Array(UInt64),\n min Int8,\n max Int8,\n sum_of_samples Int64,\n squared_mean Float64,\n p50_marker_heights Array(Float64),\n p50_marker_positions Array(UInt64),\n p50_desired_marker_positions Array(Float64),\n p90_marker_heights Array(Float64),\n p90_marker_positions Array(UInt64),\n p90_desired_marker_positions Array(Float64),\n p99_marker_heights Array(Float64),\n p99_marker_positions Array(UInt64),\n p99_desired_marker_positions Array(Float64)\n)\nENGINE = MergeTree()\nORDER BY (timeseries_name, timeseries_key, start_time, timestamp)\nTTL toDateTime(timestamp) + INTERVAL 30 DAY
30422025-04-30T06:43:51.177ZTRACtest_utilization (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 4d1485f0-110e-4161-9a33-c4610d68be5d sql = \n\nCREATE TABLE IF NOT EXISTS oximeter.measurements_histogramu8\n(\n timeseries_name String,\n timeseries_key UInt64,\n start_time DateTime64(9, 'UTC'),\n timestamp DateTime64(9, 'UTC'),\n bins Array(UInt8),\n counts Array(UInt64),\n min UInt8,\n max UInt8,\n sum_of_samples Int64,\n squared_mean Float64,\n p50_marker_heights Array(Float64),\n p50_marker_positions Array(UInt64),\n p50_desired_marker_positions Array(Float64),\n p90_marker_heights Array(Float64),\n p90_marker_positions Array(UInt64),\n p90_desired_marker_positions Array(Float64),\n p99_marker_heights Array(Float64),\n p99_marker_positions Array(UInt64),\n p99_desired_marker_positions Array(Float64)\n)\nENGINE = MergeTree()\nORDER BY (timeseries_name, timeseries_key, start_time, timestamp)\nTTL toDateTime(timestamp) + INTERVAL 30 DAY
30432025-04-30T06:43:51.182ZTRACtest_utilization (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 4d1485f0-110e-4161-9a33-c4610d68be5d sql = \n\nCREATE TABLE IF NOT EXISTS oximeter.measurements_histogrami16\n(\n timeseries_name String,\n timeseries_key UInt64,\n start_time DateTime64(9, 'UTC'),\n timestamp DateTime64(9, 'UTC'),\n bins Array(Int16),\n counts Array(UInt64),\n min Int16,\n max Int16,\n sum_of_samples Int64,\n squared_mean Float64,\n p50_marker_heights Array(Float64),\n p50_marker_positions Array(UInt64),\n p50_desired_marker_positions Array(Float64),\n p90_marker_heights Array(Float64),\n p90_marker_positions Array(UInt64),\n p90_desired_marker_positions Array(Float64),\n p99_marker_heights Array(Float64),\n p99_marker_positions Array(UInt64),\n p99_desired_marker_positions Array(Float64)\n)\nENGINE = MergeTree()\nORDER BY (timeseries_name, timeseries_key, start_time, timestamp)\nTTL toDateTime(timestamp) + INTERVAL 30 DAY
30442025-04-30T06:43:51.187ZTRACtest_utilization (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 4d1485f0-110e-4161-9a33-c4610d68be5d sql = \n\nCREATE TABLE IF NOT EXISTS oximeter.measurements_histogramu16\n(\n timeseries_name String,\n timeseries_key UInt64,\n start_time DateTime64(9, 'UTC'),\n timestamp DateTime64(9, 'UTC'),\n bins Array(UInt16),\n counts Array(UInt64),\n min UInt16,\n max UInt16,\n sum_of_samples Int64,\n squared_mean Float64,\n p50_marker_heights Array(Float64),\n p50_marker_positions Array(UInt64),\n p50_desired_marker_positions Array(Float64),\n p90_marker_heights Array(Float64),\n p90_marker_positions Array(UInt64),\n p90_desired_marker_positions Array(Float64),\n p99_marker_heights Array(Float64),\n p99_marker_positions Array(UInt64),\n p99_desired_marker_positions Array(Float64)\n)\nENGINE = MergeTree()\nORDER BY (timeseries_name, timeseries_key, start_time, timestamp)\nTTL toDateTime(timestamp) + INTERVAL 30 DAY
30452025-04-30T06:43:51.192ZTRACtest_utilization (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 4d1485f0-110e-4161-9a33-c4610d68be5d sql = \n\nCREATE TABLE IF NOT EXISTS oximeter.measurements_histogrami32\n(\n timeseries_name String,\n timeseries_key UInt64,\n start_time DateTime64(9, 'UTC'),\n timestamp DateTime64(9, 'UTC'),\n bins Array(Int32),\n counts Array(UInt64),\n min Int32,\n max Int32,\n sum_of_samples Int64,\n squared_mean Float64,\n p50_marker_heights Array(Float64),\n p50_marker_positions Array(UInt64),\n p50_desired_marker_positions Array(Float64),\n p90_marker_heights Array(Float64),\n p90_marker_positions Array(UInt64),\n p90_desired_marker_positions Array(Float64),\n p99_marker_heights Array(Float64),\n p99_marker_positions Array(UInt64),\n p99_desired_marker_positions Array(Float64)\n)\nENGINE = MergeTree()\nORDER BY (timeseries_name, timeseries_key, start_time, timestamp)\nTTL toDateTime(timestamp) + INTERVAL 30 DAY
30462025-04-30T06:43:51.196ZTRACtest_utilization (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 4d1485f0-110e-4161-9a33-c4610d68be5d sql = \n\nCREATE TABLE IF NOT EXISTS oximeter.measurements_histogramu32\n(\n timeseries_name String,\n timeseries_key UInt64,\n start_time DateTime64(9, 'UTC'),\n timestamp DateTime64(9, 'UTC'),\n bins Array(UInt32),\n counts Array(UInt64),\n min UInt32,\n max UInt32,\n sum_of_samples Int64,\n squared_mean Float64,\n p50_marker_heights Array(Float64),\n p50_marker_positions Array(UInt64),\n p50_desired_marker_positions Array(Float64),\n p90_marker_heights Array(Float64),\n p90_marker_positions Array(UInt64),\n p90_desired_marker_positions Array(Float64),\n p99_marker_heights Array(Float64),\n p99_marker_positions Array(UInt64),\n p99_desired_marker_positions Array(Float64)\n)\nENGINE = MergeTree()\nORDER BY (timeseries_name, timeseries_key, start_time, timestamp)\nTTL toDateTime(timestamp) + INTERVAL 30 DAY
30472025-04-30T06:43:51.200ZTRACtest_utilization (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 4d1485f0-110e-4161-9a33-c4610d68be5d sql = \n\nCREATE TABLE IF NOT EXISTS oximeter.measurements_histogrami64\n(\n timeseries_name String,\n timeseries_key UInt64,\n start_time DateTime64(9, 'UTC'),\n timestamp DateTime64(9, 'UTC'),\n bins Array(Int64),\n counts Array(UInt64),\n min Int64,\n max Int64,\n sum_of_samples Int64,\n squared_mean Float64,\n p50_marker_heights Array(Float64),\n p50_marker_positions Array(UInt64),\n p50_desired_marker_positions Array(Float64),\n p90_marker_heights Array(Float64),\n p90_marker_positions Array(UInt64),\n p90_desired_marker_positions Array(Float64),\n p99_marker_heights Array(Float64),\n p99_marker_positions Array(UInt64),\n p99_desired_marker_positions Array(Float64)\n)\nENGINE = MergeTree()\nORDER BY (timeseries_name, timeseries_key, start_time, timestamp)\nTTL toDateTime(timestamp) + INTERVAL 30 DAY
30482025-04-30T06:43:51.202ZDEBGtest_utilization (producer-registration-task): will register with Nexus at 127.0.0.1:32956
30492025-04-30T06:43:51.253ZDEBGtest_utilization (producer-registration-task): client request body = Some(Body) method = POST uri = http://127.0.0.1:32956/metrics/producers
30502025-04-30T06:43:51.253ZTRACtest_utilization (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 4d1485f0-110e-4161-9a33-c4610d68be5d sql = \n\nCREATE TABLE IF NOT EXISTS oximeter.measurements_histogramu64\n(\n timeseries_name String,\n timeseries_key UInt64,\n start_time DateTime64(9, 'UTC'),\n timestamp DateTime64(9, 'UTC'),\n bins Array(UInt64),\n counts Array(UInt64),\n min UInt64,\n max UInt64,\n sum_of_samples Int64,\n squared_mean Float64,\n p50_marker_heights Array(Float64),\n p50_marker_positions Array(UInt64),\n p50_desired_marker_positions Array(Float64),\n p90_marker_heights Array(Float64),\n p90_marker_positions Array(UInt64),\n p90_desired_marker_positions Array(Float64),\n p99_marker_heights Array(Float64),\n p99_marker_positions Array(UInt64),\n p99_desired_marker_positions Array(Float64)\n)\nENGINE = MergeTree()\nORDER BY (timeseries_name, timeseries_key, start_time, timestamp)\nTTL toDateTime(timestamp) + INTERVAL 30 DAY
30512025-04-30T06:43:51.253ZINFOtest_utilization (dropshot_internal): accepted connection local_addr = 127.0.0.1:32956 remote_addr = 127.0.0.1:65350
30522025-04-30T06:43:51.254ZTRACtest_utilization (dropshot_internal): incoming request local_addr = 127.0.0.1:32956 method = POST remote_addr = 127.0.0.1:65350 req_id = 1e184e36-c3b1-4765-ace6-0050f4a29d73 uri = /metrics/producers
30532025-04-30T06:43:51.254ZTRACtest_utilization (dropshot_internal): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = 127.0.0.1:32956 method = POST remote_addr = 127.0.0.1:65350 req_id = 1e184e36-c3b1-4765-ace6-0050f4a29d73 resource = Database uri = /metrics/producers
30542025-04-30T06:43:51.255ZDEBGtest_utilization (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = 127.0.0.1:32956 method = POST remote_addr = 127.0.0.1:65350 req_id = 1e184e36-c3b1-4765-ace6-0050f4a29d73 roles = RoleSet { roles: {} } uri = /metrics/producers
30552025-04-30T06:43:51.256ZDEBGtest_utilization (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = 127.0.0.1:32956 method = POST remote_addr = 127.0.0.1:65350 req_id = 1e184e36-c3b1-4765-ace6-0050f4a29d73 resource = Database result = Ok(()) uri = /metrics/producers
30562025-04-30T06:43:51.260ZTRACtest_utilization (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 4d1485f0-110e-4161-9a33-c4610d68be5d sql = \n\nCREATE TABLE IF NOT EXISTS oximeter.measurements_histogramf32\n(\n timeseries_name String,\n timeseries_key UInt64,\n start_time DateTime64(9, 'UTC'),\n timestamp DateTime64(9, 'UTC'),\n bins Array(Float32),\n counts Array(UInt64),\n min Float32,\n max Float32,\n sum_of_samples Float64,\n squared_mean Float64,\n p50_marker_heights Array(Float64),\n p50_marker_positions Array(UInt64),\n p50_desired_marker_positions Array(Float64),\n p90_marker_heights Array(Float64),\n p90_marker_positions Array(UInt64),\n p90_desired_marker_positions Array(Float64),\n p99_marker_heights Array(Float64),\n p99_marker_positions Array(UInt64),\n p99_desired_marker_positions Array(Float64)\n)\nENGINE = MergeTree()\nORDER BY (timeseries_name, timeseries_key, start_time, timestamp)\nTTL toDateTime(timestamp) + INTERVAL 30 DAY
30572025-04-30T06:43:51.262ZINFOtest_utilization (dropshot_internal): request completed error_message_external = Service Unavailable error_message_internal = no Oximeter instances available for assignment latency_us = 7574 local_addr = 127.0.0.1:32956 method = POST remote_addr = 127.0.0.1:65350 req_id = 1e184e36-c3b1-4765-ace6-0050f4a29d73 response_code = 503 uri = /metrics/producers
30582025-04-30T06:43:51.262ZDEBGtest_utilization (producer-registration-task): client response result = Ok(Response { url: "http://127.0.0.1:32956/metrics/producers", status: 503, headers: {"content-type": "application/json", "x-request-id": "1e184e36-c3b1-4765-ace6-0050f4a29d73", "content-length": "133", "date": "Wed, 30 Apr 2025 06:43:50 GMT"} })
30592025-04-30T06:43:51.265ZWARNtest_utilization (producer-registration-task): failed to register with Nexus, will retry count = 1 delay = 351.961901ms error = "Error Response: status: 503 Service Unavailable; headers: {\\"content-type\\": \\"application/json\\", \\"x-request-id\\": \\"1e184e36-c3b1-4765-ace6-0050f4a29d73\\", \\"content-length\\": \\"133\\", \\"date\\": \\"Wed, 30 Apr 2025 06:43:50 GMT\\"}; value: Error { error_code: Some(\\"ServiceNotAvailable\\"), message: \\"Service Unavailable\\", request_id: \\"1e184e36-c3b1-4765-ace6-0050f4a29d73\\" }" file = oximeter/producer/src/lib.rs:371
30602025-04-30T06:43:51.266ZTRACtest_utilization (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 4d1485f0-110e-4161-9a33-c4610d68be5d sql = \n\nCREATE TABLE IF NOT EXISTS oximeter.measurements_histogramf64\n(\n timeseries_name String,\n timeseries_key UInt64,\n start_time DateTime64(9, 'UTC'),\n timestamp DateTime64(9, 'UTC'),\n bins Array(Float64),\n counts Array(UInt64),\n min Float64,\n max Float64,\n sum_of_samples Float64,\n squared_mean Float64,\n p50_marker_heights Array(Float64),\n p50_marker_positions Array(UInt64),\n p50_desired_marker_positions Array(Float64),\n p90_marker_heights Array(Float64),\n p90_marker_positions Array(UInt64),\n p90_desired_marker_positions Array(Float64),\n p99_marker_heights Array(Float64),\n p99_marker_positions Array(UInt64),\n p99_desired_marker_positions Array(Float64)\n)\nENGINE = MergeTree()\nORDER BY (timeseries_name, timeseries_key, start_time, timestamp)\nTTL toDateTime(timestamp) + INTERVAL 30 DAY
30612025-04-30T06:43:51.281ZTRACtest_utilization (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 4d1485f0-110e-4161-9a33-c4610d68be5d sql = \n\n/* The field tables store named dimensions of each timeseries.\n *\n * As with the measurement tables, there is one field table for each field data\n * type. Fields are deduplicated by using the "replacing merge tree", though\n * this behavior **must not** be relied upon for query correctness.\n *\n * The index for the fields differs from the measurements, however. Rows are\n * sorted by timeseries name, then field name, field value, and finally\n * timeseries key. This reflects the most common pattern for looking them up:\n * by field name and possibly value, within a timeseries. The resulting keys are\n * usually then used to look up measurements.\n *\n * NOTE: We may want to consider a secondary index on these tables, sorting by\n * timeseries name and then key, since it would improve lookups where one\n * already has the key. Realistically though, these tables are quite small and\n * so performance benefits will be low in absolute terms.\n *\n * TTL: We use a materialized column to expire old field table records. This\n * column is generated automatically by the database whenever a new row is\n * inserted. It cannot be inserted directly, nor is it returned in a `SELECT *`\n * query. Since these tables are `ReplacingMergeTree`s, that means the last\n * record will remain during a deduplication, which will have the last\n * timestamp. ClickHouse will then expire old data for us, similar to the\n * measurement tables.\n */\nCREATE TABLE IF NOT EXISTS oximeter.fields_bool\n(\n timeseries_name String,\n timeseries_key UInt64,\n field_name String,\n field_value Bool,\n last_updated_at DateTime MATERIALIZED now()\n)\nENGINE = ReplacingMergeTree()\nORDER BY (timeseries_name, field_name, field_value, timeseries_key)\nTTL last_updated_at + INTERVAL 30 DAY
30622025-04-30T06:43:51.285ZTRACtest_utilization (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 4d1485f0-110e-4161-9a33-c4610d68be5d sql = \n\nCREATE TABLE IF NOT EXISTS oximeter.fields_i8\n(\n timeseries_name String,\n timeseries_key UInt64,\n field_name String,\n field_value Int8,\n last_updated_at DateTime MATERIALIZED now()\n)\nENGINE = ReplacingMergeTree()\nORDER BY (timeseries_name, field_name, field_value, timeseries_key)\nTTL last_updated_at + INTERVAL 30 DAY
30632025-04-30T06:43:51.289ZTRACtest_utilization (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 4d1485f0-110e-4161-9a33-c4610d68be5d sql = \n\nCREATE TABLE IF NOT EXISTS oximeter.fields_u8\n(\n timeseries_name String,\n timeseries_key UInt64,\n field_name String,\n field_value UInt8,\n last_updated_at DateTime MATERIALIZED now()\n)\nENGINE = ReplacingMergeTree()\nORDER BY (timeseries_name, field_name, field_value, timeseries_key)\nTTL last_updated_at + INTERVAL 30 DAY
30642025-04-30T06:43:51.294ZTRACtest_utilization (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 4d1485f0-110e-4161-9a33-c4610d68be5d sql = \n\nCREATE TABLE IF NOT EXISTS oximeter.fields_i16\n(\n timeseries_name String,\n timeseries_key UInt64,\n field_name String,\n field_value Int16,\n last_updated_at DateTime MATERIALIZED now()\n)\nENGINE = ReplacingMergeTree()\nORDER BY (timeseries_name, field_name, field_value, timeseries_key)\nTTL last_updated_at + INTERVAL 30 DAY
30652025-04-30T06:43:51.298ZTRACtest_utilization (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 4d1485f0-110e-4161-9a33-c4610d68be5d sql = \n\nCREATE TABLE IF NOT EXISTS oximeter.fields_u16\n(\n timeseries_name String,\n timeseries_key UInt64,\n field_name String,\n field_value UInt16,\n last_updated_at DateTime MATERIALIZED now()\n)\nENGINE = ReplacingMergeTree()\nORDER BY (timeseries_name, field_name, field_value, timeseries_key)\nTTL last_updated_at + INTERVAL 30 DAY
30662025-04-30T06:43:51.302ZTRACtest_utilization (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 4d1485f0-110e-4161-9a33-c4610d68be5d sql = \n\nCREATE TABLE IF NOT EXISTS oximeter.fields_i32\n(\n timeseries_name String,\n timeseries_key UInt64,\n field_name String,\n field_value Int32,\n last_updated_at DateTime MATERIALIZED now()\n)\nENGINE = ReplacingMergeTree()\nORDER BY (timeseries_name, field_name, field_value, timeseries_key)\nTTL last_updated_at + INTERVAL 30 DAY
30672025-04-30T06:43:51.307ZWARNtest_utilization (datastore): Non-retryable transaction failure err = DatabaseError(UniqueViolation, "duplicate key value violates unique constraint \\"inv_caboose_pkey\\"") transaction = inventory_insert_collection
30682025-04-30T06:43:51.308ZWARNtest_utilization (BackgroundTasks): inventory collection failed background_task = inventory_collection error = failed to collect inventory: saving inventory to database: Internal Error: unexpected database error: duplicate key value violates unique constraint "inv_caboose_pkey"
30692025-04-30T06:43:51.308ZDEBGtest_utilization (BackgroundTasks): activation complete background_task = inventory_collection elapsed = 2.1770696s iteration = 3 status = {"error":"failed to collect inventory: saving inventory to database: Internal Error: unexpected database error: duplicate key value violates unique constraint \\"inv_caboose_pkey\\""}
30702025-04-30T06:43:51.309ZTRACtest_utilization (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 4d1485f0-110e-4161-9a33-c4610d68be5d sql = \n\nCREATE TABLE IF NOT EXISTS oximeter.fields_u32\n(\n timeseries_name String,\n timeseries_key UInt64,\n field_name String,\n field_value UInt32,\n last_updated_at DateTime MATERIALIZED now()\n)\nENGINE = ReplacingMergeTree()\nORDER BY (timeseries_name, field_name, field_value, timeseries_key)\nTTL last_updated_at + INTERVAL 30 DAY
30712025-04-30T06:43:51.315ZTRACtest_utilization (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 4d1485f0-110e-4161-9a33-c4610d68be5d sql = \n\nCREATE TABLE IF NOT EXISTS oximeter.fields_i64\n(\n timeseries_name String,\n timeseries_key UInt64,\n field_name String,\n field_value Int64,\n last_updated_at DateTime MATERIALIZED now()\n)\nENGINE = ReplacingMergeTree()\nORDER BY (timeseries_name, field_name, field_value, timeseries_key)\nTTL last_updated_at + INTERVAL 30 DAY
30722025-04-30T06:43:51.322ZTRACtest_utilization (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 4d1485f0-110e-4161-9a33-c4610d68be5d sql = \n\nCREATE TABLE IF NOT EXISTS oximeter.fields_u64\n(\n timeseries_name String,\n timeseries_key UInt64,\n field_name String,\n field_value UInt64,\n last_updated_at DateTime MATERIALIZED now()\n)\nENGINE = ReplacingMergeTree()\nORDER BY (timeseries_name, field_name, field_value, timeseries_key)\nTTL last_updated_at + INTERVAL 30 DAY
30732025-04-30T06:43:51.332ZTRACtest_utilization (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 4d1485f0-110e-4161-9a33-c4610d68be5d sql = \n\nCREATE TABLE IF NOT EXISTS oximeter.fields_ipaddr\n(\n timeseries_name String,\n timeseries_key UInt64,\n field_name String,\n field_value IPv6,\n last_updated_at DateTime MATERIALIZED now()\n)\nENGINE = ReplacingMergeTree()\nORDER BY (timeseries_name, field_name, field_value, timeseries_key)\nTTL last_updated_at + INTERVAL 30 DAY
30742025-04-30T06:43:51.335ZTRACtest_utilization (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 4d1485f0-110e-4161-9a33-c4610d68be5d sql = \n\nCREATE TABLE IF NOT EXISTS oximeter.fields_string\n(\n timeseries_name String,\n timeseries_key UInt64,\n field_name String,\n field_value String,\n last_updated_at DateTime MATERIALIZED now()\n)\nENGINE = ReplacingMergeTree()\nORDER BY (timeseries_name, field_name, field_value, timeseries_key)\nTTL last_updated_at + INTERVAL 30 DAY
30752025-04-30T06:43:51.338ZTRACtest_utilization (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 4d1485f0-110e-4161-9a33-c4610d68be5d sql = \n\nCREATE TABLE IF NOT EXISTS oximeter.fields_uuid\n(\n timeseries_name String,\n timeseries_key UInt64,\n field_name String,\n field_value UUID,\n last_updated_at DateTime MATERIALIZED now()\n)\nENGINE = ReplacingMergeTree()\nORDER BY (timeseries_name, field_name, field_value, timeseries_key)\nTTL last_updated_at + INTERVAL 30 DAY
30762025-04-30T06:43:51.343ZTRACtest_utilization (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 4d1485f0-110e-4161-9a33-c4610d68be5d sql = \n\n/* The timeseries schema table stores the extracted schema for the samples\n * oximeter collects.\n */\nCREATE TABLE IF NOT EXISTS oximeter.timeseries_schema\n(\n timeseries_name String,\n fields Nested(\n name String,\n type Enum(\n 'Bool' = 1,\n 'I64' = 2,\n 'IpAddr' = 3,\n 'String' = 4,\n 'Uuid' = 6,\n 'I8' = 7,\n 'U8' = 8,\n 'I16' = 9,\n 'U16' = 10,\n 'I32' = 11,\n 'U32' = 12,\n 'U64' = 13\n ),\n source Enum(\n 'Target' = 1,\n 'Metric' = 2\n )\n ),\n datum_type Enum(\n 'Bool' = 1,\n 'I64' = 2,\n 'F64' = 3,\n 'String' = 4,\n 'Bytes' = 5,\n 'CumulativeI64' = 6,\n 'CumulativeF64' = 7,\n 'HistogramI64' = 8,\n 'HistogramF64' = 9,\n 'I8' = 10,\n 'U8' = 11,\n 'I16' = 12,\n 'U16' = 13,\n 'I32' = 14,\n 'U32' = 15,\n 'U64' = 16,\n 'F32' = 17,\n 'CumulativeU64' = 18,\n 'CumulativeF32' = 19,\n 'HistogramI8' = 20,\n 'HistogramU8' = 21,\n 'HistogramI16' = 22,\n 'HistogramU16' = 23,\n 'HistogramI32' = 24,\n 'HistogramU32' = 25,\n 'HistogramU64' = 26,\n 'HistogramF32' = 27\n ),\n created DateTime64(9, 'UTC')\n)\nENGINE = MergeTree()\nORDER BY (timeseries_name, fields.name)
30772025-04-30T06:43:51.347ZINFOtest_utilization (clickhouse-client): inserting current version collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 file = oximeter/db/src/client/mod.rs:825 id = 4d1485f0-110e-4161-9a33-c4610d68be5d version = 14
30782025-04-30T06:43:51.347ZTRACtest_utilization (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 4d1485f0-110e-4161-9a33-c4610d68be5d sql = INSERT INTO oximeter.version (*) VALUES (14, now());
30792025-04-30T06:43:51.353ZDEBGtest_utilization (clickhouse-client): registered USDT probes collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 57850f82-85cb-49cf-9e15-eb30a7ec87b9
30802025-04-30T06:43:51.354ZDEBGtest_utilization (dropshot): registered endpoint local_addr = [::1]:37473 method = GET path = /info versions = all
30812025-04-30T06:43:51.354ZDEBGtest_utilization (dropshot): registered endpoint local_addr = [::1]:37473 method = GET path = /producers versions = all
30822025-04-30T06:43:51.354ZDEBGtest_utilization (dropshot): registered endpoint local_addr = [::1]:37473 method = DELETE path = /producers/{producer_id} versions = all
30832025-04-30T06:43:51.354ZDEBGtest_utilization (dropshot): registered endpoint local_addr = [::1]:37473 method = GET path = /producers/{producer_id} versions = all
30842025-04-30T06:43:51.354ZINFOtest_utilization (dropshot): listening file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:328 local_addr = [::1]:37473
30852025-04-30T06:43:51.354ZDEBGtest_utilization (dropshot): successfully registered DTrace USDT probes local_addr = [::1]:37473
30862025-04-30T06:43:51.355ZDEBGtest_utilization (oximeter): registered USDT probes
30872025-04-30T06:43:51.355ZDEBGtest_utilization (oximeter): contacting nexus
30882025-04-30T06:43:51.815ZDEBGtest_utilization (oximeter): client request body = Some(Body) method = POST uri = http://127.0.0.1:32956/metrics/collectors
30892025-04-30T06:43:51.816ZDEBGtest_utilization (producer-registration-task): will register with Nexus at 127.0.0.1:32956
30902025-04-30T06:43:51.926ZDEBGtest_utilization (producer-registration-task): client request body = Some(Body) method = POST uri = http://127.0.0.1:32956/metrics/producers
30912025-04-30T06:43:51.928ZINFOtest_utilization (dropshot_internal): accepted connection local_addr = 127.0.0.1:32956 remote_addr = 127.0.0.1:55797
30922025-04-30T06:43:51.928ZINFOtest_utilization (dropshot_internal): accepted connection local_addr = 127.0.0.1:32956 remote_addr = 127.0.0.1:63177
30932025-04-30T06:43:51.929ZTRACtest_utilization (dropshot_internal): incoming request local_addr = 127.0.0.1:32956 method = POST remote_addr = 127.0.0.1:63177 req_id = 4b121ce1-1a7b-4980-9027-843cd833e8a2 uri = /metrics/producers
30942025-04-30T06:43:51.930ZTRACtest_utilization (dropshot_internal): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = 127.0.0.1:32956 method = POST remote_addr = 127.0.0.1:63177 req_id = 4b121ce1-1a7b-4980-9027-843cd833e8a2 resource = Database uri = /metrics/producers
30952025-04-30T06:43:51.930ZDEBGtest_utilization (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = 127.0.0.1:32956 method = POST remote_addr = 127.0.0.1:63177 req_id = 4b121ce1-1a7b-4980-9027-843cd833e8a2 roles = RoleSet { roles: {} } uri = /metrics/producers
30962025-04-30T06:43:51.931ZDEBGtest_utilization (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = 127.0.0.1:32956 method = POST remote_addr = 127.0.0.1:63177 req_id = 4b121ce1-1a7b-4980-9027-843cd833e8a2 resource = Database result = Ok(()) uri = /metrics/producers
30972025-04-30T06:43:51.933ZTRACtest_utilization (dropshot_internal): incoming request local_addr = 127.0.0.1:32956 method = POST remote_addr = 127.0.0.1:55797 req_id = fc16f7ad-8c31-470f-a758-b9d56e1b411e uri = /metrics/collectors
30982025-04-30T06:43:51.933ZTRACtest_utilization (dropshot_internal): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = 127.0.0.1:32956 method = POST remote_addr = 127.0.0.1:55797 req_id = fc16f7ad-8c31-470f-a758-b9d56e1b411e resource = Database uri = /metrics/collectors
30992025-04-30T06:43:51.933ZDEBGtest_utilization (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = 127.0.0.1:32956 method = POST remote_addr = 127.0.0.1:55797 req_id = fc16f7ad-8c31-470f-a758-b9d56e1b411e roles = RoleSet { roles: {} } uri = /metrics/collectors
31002025-04-30T06:43:51.938ZDEBGtest_utilization (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = 127.0.0.1:32956 method = POST remote_addr = 127.0.0.1:55797 req_id = fc16f7ad-8c31-470f-a758-b9d56e1b411e resource = Database result = Ok(()) uri = /metrics/collectors
31012025-04-30T06:43:51.945ZINFOtest_utilization (dropshot_internal): request completed error_message_external = Service Unavailable error_message_internal = no Oximeter instances available for assignment latency_us = 15441 local_addr = 127.0.0.1:32956 method = POST remote_addr = 127.0.0.1:63177 req_id = 4b121ce1-1a7b-4980-9027-843cd833e8a2 response_code = 503 uri = /metrics/producers
31022025-04-30T06:43:51.946ZDEBGtest_utilization (producer-registration-task): client response result = Ok(Response { url: "http://127.0.0.1:32956/metrics/producers", status: 503, headers: {"content-type": "application/json", "x-request-id": "4b121ce1-1a7b-4980-9027-843cd833e8a2", "content-length": "133", "date": "Wed, 30 Apr 2025 06:43:51 GMT"} })
31032025-04-30T06:43:51.947ZWARNtest_utilization (producer-registration-task): failed to register with Nexus, will retry count = 2 delay = 1.033944858s error = "Error Response: status: 503 Service Unavailable; headers: {\\"content-type\\": \\"application/json\\", \\"x-request-id\\": \\"4b121ce1-1a7b-4980-9027-843cd833e8a2\\", \\"content-length\\": \\"133\\", \\"date\\": \\"Wed, 30 Apr 2025 06:43:51 GMT\\"}; value: Error { error_code: Some(\\"ServiceNotAvailable\\"), message: \\"Service Unavailable\\", request_id: \\"4b121ce1-1a7b-4980-9027-843cd833e8a2\\" }" file = oximeter/producer/src/lib.rs:371
31042025-04-30T06:43:51.948ZINFOtest_utilization (nexus): registered new oximeter metric collection server address = [::1]:37473 collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78
31052025-04-30T06:43:51.948ZINFOtest_utilization (dropshot_internal): request completed latency_us = 15242 local_addr = 127.0.0.1:32956 method = POST remote_addr = 127.0.0.1:55797 req_id = fc16f7ad-8c31-470f-a758-b9d56e1b411e response_code = 204 uri = /metrics/collectors
31062025-04-30T06:43:51.951ZDEBGtest_utilization (oximeter): client response result = Ok(Response { url: "http://127.0.0.1:32956/metrics/collectors", status: 204, headers: {"x-request-id": "fc16f7ad-8c31-470f-a758-b9d56e1b411e", "date": "Wed, 30 Apr 2025 06:43:51 GMT"} })
31072025-04-30T06:43:51.951ZINFOtest_utilization (oximeter): oximeter registered with nexus file = oximeter/collector/src/lib.rs:392 id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78
31082025-04-30T06:43:51.951ZDEBGtest_utilization (init_with_steps): Running step start_producer_server
31092025-04-30T06:43:51.951ZDEBGtest_utilization: Starting test metric Producer Server
31102025-04-30T06:43:51.954ZINFOtest_utilization (external client test context): client request body = Body { inner: BoxBody } method = POST uri = http://127.0.0.1:46348/v1/system/ip-pools
31112025-04-30T06:43:51.955ZINFOtest_utilization (oximeter-agent): starting refresh list task collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 file = oximeter/collector/src/agent.rs:559
31122025-04-30T06:43:52.011ZINFOtest_utilization (dropshot_internal): accepted connection local_addr = 127.0.0.1:32956 remote_addr = 127.0.0.1:56440
31132025-04-30T06:43:52.011ZINFOtest_utilization (dropshot_external): accepted connection local_addr = 127.0.0.1:46348 remote_addr = 127.0.0.1:41283
31142025-04-30T06:43:52.011ZINFOtest_utilization (oximeter-agent): refreshing list of producers from Nexus collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 file = oximeter/collector/src/agent.rs:562
31152025-04-30T06:43:52.015ZDEBGtest_utilization (oximeter): client request body = None method = GET uri = http://127.0.0.1:32956/metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?sort_by=id_ascending
31162025-04-30T06:43:52.016ZTRACtest_utilization (dropshot_internal): incoming request local_addr = 127.0.0.1:32956 method = GET remote_addr = 127.0.0.1:55797 req_id = ab5f3eb2-4a60-4cd2-bf12-ce4ff43e11c3 uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?sort_by=id_ascending
31172025-04-30T06:43:52.016ZTRACtest_utilization (dropshot_internal): incoming request local_addr = 127.0.0.1:32956 method = POST remote_addr = 127.0.0.1:56440 req_id = a8e99988-464b-4f81-9f0f-0d74ef20d83a uri = /metrics/producers
31182025-04-30T06:43:52.017ZTRACtest_utilization (dropshot_internal): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = 127.0.0.1:32956 method = GET remote_addr = 127.0.0.1:55797 req_id = ab5f3eb2-4a60-4cd2-bf12-ce4ff43e11c3 resource = Database uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?sort_by=id_ascending
31192025-04-30T06:43:52.017ZDEBGtest_utilization (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = 127.0.0.1:32956 method = GET remote_addr = 127.0.0.1:55797 req_id = ab5f3eb2-4a60-4cd2-bf12-ce4ff43e11c3 roles = RoleSet { roles: {} } uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?sort_by=id_ascending
31202025-04-30T06:43:52.018ZDEBGtest_utilization (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = 127.0.0.1:32956 method = GET remote_addr = 127.0.0.1:55797 req_id = ab5f3eb2-4a60-4cd2-bf12-ce4ff43e11c3 resource = Database result = Ok(()) uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?sort_by=id_ascending
31212025-04-30T06:43:52.019ZTRACtest_utilization (dropshot_internal): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = 127.0.0.1:32956 method = POST remote_addr = 127.0.0.1:56440 req_id = a8e99988-464b-4f81-9f0f-0d74ef20d83a resource = Database uri = /metrics/producers
31222025-04-30T06:43:52.019ZDEBGtest_utilization (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = 127.0.0.1:32956 method = POST remote_addr = 127.0.0.1:56440 req_id = a8e99988-464b-4f81-9f0f-0d74ef20d83a roles = RoleSet { roles: {} } uri = /metrics/producers
31232025-04-30T06:43:52.020ZDEBGtest_utilization (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = 127.0.0.1:32956 method = POST remote_addr = 127.0.0.1:56440 req_id = a8e99988-464b-4f81-9f0f-0d74ef20d83a resource = Database result = Ok(()) uri = /metrics/producers
31242025-04-30T06:43:52.020ZTRACtest_utilization (dropshot_external): incoming request local_addr = 127.0.0.1:46348 method = POST remote_addr = 127.0.0.1:41283 req_id = 404787a2-34c9-42db-b37a-7e19aaefbb9d uri = /v1/system/ip-pools
31252025-04-30T06:43:52.022ZTRACtest_utilization (dropshot_external): authn: trying SchemeName("spoof") local_addr = 127.0.0.1:46348 method = POST remote_addr = 127.0.0.1:41283 req_id = 404787a2-34c9-42db-b37a-7e19aaefbb9d uri = /v1/system/ip-pools
31262025-04-30T06:43:52.022ZDEBGtest_utilization (dropshot_external): looking up silo for user local_addr = 127.0.0.1:46348 method = POST remote_addr = 127.0.0.1:41283 req_id = 404787a2-34c9-42db-b37a-7e19aaefbb9d silo_user_id = 001de000-05e4-4000-8000-000000004007 uri = /v1/system/ip-pools
31272025-04-30T06:43:52.022ZTRACtest_utilization (ExternalAuthn): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database
31282025-04-30T06:43:52.022ZDEBGtest_utilization (ExternalAuthn): roles roles = RoleSet { roles: {} }
31292025-04-30T06:43:52.023ZDEBGtest_utilization (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
31302025-04-30T06:43:52.034ZINFOtest_utilization (dropshot_internal): request completed latency_us = 18241 local_addr = 127.0.0.1:32956 method = GET remote_addr = 127.0.0.1:55797 req_id = ab5f3eb2-4a60-4cd2-bf12-ce4ff43e11c3 response_code = 200 uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?sort_by=id_ascending
31312025-04-30T06:43:52.035ZTRACtest_utilization (ExternalAuthn): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database
31322025-04-30T06:43:52.035ZDEBGtest_utilization (ExternalAuthn): roles roles = RoleSet { roles: {} }
31332025-04-30T06:43:52.036ZDEBGtest_utilization (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
31342025-04-30T06:43:52.039ZDEBGtest_utilization (oximeter): client response result = Ok(Response { url: "http://127.0.0.1:32956/metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?sort_by=id_ascending", status: 200, headers: {"content-type": "application/json", "x-request-id": "ab5f3eb2-4a60-4cd2-bf12-ce4ff43e11c3", "content-length": "29", "date": "Wed, 30 Apr 2025 06:43:51 GMT"} })
31352025-04-30T06:43:52.040ZINFOtest_utilization (oximeter-agent): refreshed list of producers from Nexus collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 file = oximeter/collector/src/agent.rs:637 n_current_tasks = 0 n_pruned_tasks = 0
31362025-04-30T06:43:52.043ZTRACtest_utilization (ExternalAuthn): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = SiloUser { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000000, lookup_type: ById(001de000-5110-4000-8000-000000000000) }, key: 001de000-05e4-4000-8000-000000004007, lookup_type: ById(001de000-05e4-4000-8000-000000004007) }
31372025-04-30T06:43:52.043ZTRACtest_utilization (ExternalAuthn): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. } resource_id = 001de000-5110-4000-8000-000000000000 resource_type = Silo
31382025-04-30T06:43:52.043ZTRACtest_utilization (ExternalAuthn): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database
31392025-04-30T06:43:52.043ZDEBGtest_utilization (ExternalAuthn): roles roles = RoleSet { roles: {} }
31402025-04-30T06:43:52.044ZDEBGtest_utilization (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
31412025-04-30T06:43:52.051ZTRACtest_utilization (ExternalAuthn): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
31422025-04-30T06:43:52.051ZTRACtest_utilization (ExternalAuthn): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database
31432025-04-30T06:43:52.051ZDEBGtest_utilization (ExternalAuthn): roles roles = RoleSet { roles: {} }
31442025-04-30T06:43:52.052ZDEBGtest_utilization (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
31452025-04-30T06:43:52.055ZDEBGtest_utilization (ExternalAuthn): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "external-authenticator")} }
31462025-04-30T06:43:52.058ZDEBGtest_utilization (ExternalAuthn): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = SiloUser { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000000, lookup_type: ById(001de000-5110-4000-8000-000000000000) }, key: 001de000-05e4-4000-8000-000000004007, lookup_type: ById(001de000-05e4-4000-8000-000000004007) } result = Ok(())
31472025-04-30T06:43:52.058ZTRACtest_utilization (ExternalAuthn): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database
31482025-04-30T06:43:52.058ZDEBGtest_utilization (ExternalAuthn): roles roles = RoleSet { roles: {} }
31492025-04-30T06:43:52.059ZDEBGtest_utilization (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
31502025-04-30T06:43:52.063ZTRACtest_utilization (ExternalAuthn): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000000, lookup_type: ById(001de000-5110-4000-8000-000000000000) }
31512025-04-30T06:43:52.063ZTRACtest_utilization (ExternalAuthn): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. } resource_id = 001de000-5110-4000-8000-000000000000 resource_type = Silo
31522025-04-30T06:43:52.063ZTRACtest_utilization (ExternalAuthn): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database
31532025-04-30T06:43:52.063ZDEBGtest_utilization (ExternalAuthn): roles roles = RoleSet { roles: {} }
31542025-04-30T06:43:52.064ZDEBGtest_utilization (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
31552025-04-30T06:43:52.065ZINFOtest_utilization (nexus): assigned collector to new producer collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
31562025-04-30T06:43:52.065ZINFOtest_utilization (dropshot_internal): request completed latency_us = 48605 local_addr = 127.0.0.1:32956 method = POST remote_addr = 127.0.0.1:56440 req_id = a8e99988-464b-4f81-9f0f-0d74ef20d83a response_code = 201 uri = /metrics/producers
31572025-04-30T06:43:52.067ZTRACtest_utilization (ExternalAuthn): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
31582025-04-30T06:43:52.067ZTRACtest_utilization (ExternalAuthn): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database
31592025-04-30T06:43:52.067ZDEBGtest_utilization (ExternalAuthn): roles roles = RoleSet { roles: {} }
31602025-04-30T06:43:52.067ZDEBGtest_utilization (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
31612025-04-30T06:43:52.075ZDEBGtest_utilization (ExternalAuthn): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "external-authenticator")} }
31622025-04-30T06:43:52.078ZDEBGtest_utilization (ExternalAuthn): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000000, lookup_type: ById(001de000-5110-4000-8000-000000000000) } result = Ok(())
31632025-04-30T06:43:52.078ZTRACtest_utilization (dropshot_external): authn result: Ok(Context { kind: Authenticated(Details { actor: Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. } }, Some(SiloAuthnPolicy { mapped_fleet_roles: {} })), schemes_tried: [SchemeName("spoof")] }) local_addr = 127.0.0.1:46348 method = POST remote_addr = 127.0.0.1:41283 req_id = 404787a2-34c9-42db-b37a-7e19aaefbb9d uri = /v1/system/ip-pools
31642025-04-30T06:43:52.078ZTRACtest_utilization (dropshot_external): authorize begin action = CreateChild actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true local_addr = 127.0.0.1:46348 method = POST remote_addr = 127.0.0.1:41283 req_id = 404787a2-34c9-42db-b37a-7e19aaefbb9d resource = IpPoolList uri = /v1/system/ip-pools
31652025-04-30T06:43:52.078ZTRACtest_utilization (dropshot_external): loading roles actor = Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. } actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true local_addr = 127.0.0.1:46348 method = POST remote_addr = 127.0.0.1:41283 req_id = 404787a2-34c9-42db-b37a-7e19aaefbb9d resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet uri = /v1/system/ip-pools
31662025-04-30T06:43:52.078ZTRACtest_utilization (dropshot_external): authorize begin action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true local_addr = 127.0.0.1:46348 method = POST remote_addr = 127.0.0.1:41283 req_id = 404787a2-34c9-42db-b37a-7e19aaefbb9d resource = Database uri = /v1/system/ip-pools
31672025-04-30T06:43:52.078ZDEBGtest_utilization (dropshot_external): roles actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true local_addr = 127.0.0.1:46348 method = POST remote_addr = 127.0.0.1:41283 req_id = 404787a2-34c9-42db-b37a-7e19aaefbb9d roles = RoleSet { roles: {} } uri = /v1/system/ip-pools
31682025-04-30T06:43:52.079ZDEBGtest_utilization (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true local_addr = 127.0.0.1:46348 method = POST remote_addr = 127.0.0.1:41283 req_id = 404787a2-34c9-42db-b37a-7e19aaefbb9d resource = Database result = Ok(()) uri = /v1/system/ip-pools
31692025-04-30T06:43:52.093ZDEBGtest_utilization (dropshot_external): roles actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true local_addr = 127.0.0.1:46348 method = POST remote_addr = 127.0.0.1:41283 req_id = 404787a2-34c9-42db-b37a-7e19aaefbb9d roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} } uri = /v1/system/ip-pools
31702025-04-30T06:43:52.097ZDEBGtest_utilization (dropshot_external): authorize result action = CreateChild actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true local_addr = 127.0.0.1:46348 method = POST remote_addr = 127.0.0.1:41283 req_id = 404787a2-34c9-42db-b37a-7e19aaefbb9d resource = IpPoolList result = Ok(()) uri = /v1/system/ip-pools
31712025-04-30T06:43:52.097ZTRACtest_utilization (dropshot_external): authorize begin action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true local_addr = 127.0.0.1:46348 method = POST remote_addr = 127.0.0.1:41283 req_id = 404787a2-34c9-42db-b37a-7e19aaefbb9d resource = Database uri = /v1/system/ip-pools
31722025-04-30T06:43:52.097ZDEBGtest_utilization (dropshot_external): roles actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true local_addr = 127.0.0.1:46348 method = POST remote_addr = 127.0.0.1:41283 req_id = 404787a2-34c9-42db-b37a-7e19aaefbb9d roles = RoleSet { roles: {} } uri = /v1/system/ip-pools
31732025-04-30T06:43:52.097ZDEBGtest_utilization (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true local_addr = 127.0.0.1:46348 method = POST remote_addr = 127.0.0.1:41283 req_id = 404787a2-34c9-42db-b37a-7e19aaefbb9d resource = Database result = Ok(()) uri = /v1/system/ip-pools
31742025-04-30T06:43:52.104ZINFOtest_utilization (dropshot_external): request completed latency_us = 83866 local_addr = 127.0.0.1:46348 method = POST remote_addr = 127.0.0.1:41283 req_id = 404787a2-34c9-42db-b37a-7e19aaefbb9d response_code = 201 uri = /v1/system/ip-pools
31752025-04-30T06:43:52.105ZINFOtest_utilization (external client test context): client received response status = 201
31762025-04-30T06:43:52.106ZINFOtest_utilization (external client test context): client request body = Body { inner: BoxBody } method = POST uri = http://127.0.0.1:46348/v1/system/ip-pools/default/ranges/add
31772025-04-30T06:43:52.107ZTRACtest_utilization (dropshot_external): incoming request local_addr = 127.0.0.1:46348 method = POST remote_addr = 127.0.0.1:41283 req_id = dedd9d24-ec89-45d6-bdbf-e4c3b86a030c uri = /v1/system/ip-pools/default/ranges/add
31782025-04-30T06:43:52.108ZTRACtest_utilization (dropshot_external): authn: trying SchemeName("spoof") local_addr = 127.0.0.1:46348 method = POST remote_addr = 127.0.0.1:41283 req_id = dedd9d24-ec89-45d6-bdbf-e4c3b86a030c uri = /v1/system/ip-pools/default/ranges/add
31792025-04-30T06:43:52.108ZDEBGtest_utilization (dropshot_external): looking up silo for user local_addr = 127.0.0.1:46348 method = POST remote_addr = 127.0.0.1:41283 req_id = dedd9d24-ec89-45d6-bdbf-e4c3b86a030c silo_user_id = 001de000-05e4-4000-8000-000000004007 uri = /v1/system/ip-pools/default/ranges/add
31802025-04-30T06:43:52.108ZTRACtest_utilization (ExternalAuthn): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database
31812025-04-30T06:43:52.108ZDEBGtest_utilization (ExternalAuthn): roles roles = RoleSet { roles: {} }
31822025-04-30T06:43:52.109ZDEBGtest_utilization (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
31832025-04-30T06:43:52.113ZTRACtest_utilization (ExternalAuthn): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database
31842025-04-30T06:43:52.113ZDEBGtest_utilization (ExternalAuthn): roles roles = RoleSet { roles: {} }
31852025-04-30T06:43:52.118ZDEBGtest_utilization (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
31862025-04-30T06:43:52.122ZTRACtest_utilization (ExternalAuthn): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = SiloUser { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000000, lookup_type: ById(001de000-5110-4000-8000-000000000000) }, key: 001de000-05e4-4000-8000-000000004007, lookup_type: ById(001de000-05e4-4000-8000-000000004007) }
31872025-04-30T06:43:52.122ZTRACtest_utilization (ExternalAuthn): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. } resource_id = 001de000-5110-4000-8000-000000000000 resource_type = Silo
31882025-04-30T06:43:52.122ZTRACtest_utilization (ExternalAuthn): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database
31892025-04-30T06:43:52.122ZDEBGtest_utilization (ExternalAuthn): roles roles = RoleSet { roles: {} }
31902025-04-30T06:43:52.123ZDEBGtest_utilization (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
31912025-04-30T06:43:52.125ZTRACtest_utilization (ExternalAuthn): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
31922025-04-30T06:43:52.125ZTRACtest_utilization (ExternalAuthn): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database
31932025-04-30T06:43:52.125ZDEBGtest_utilization (ExternalAuthn): roles roles = RoleSet { roles: {} }
31942025-04-30T06:43:52.126ZDEBGtest_utilization (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
31952025-04-30T06:43:52.128ZDEBGtest_utilization (ExternalAuthn): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "external-authenticator")} }
31962025-04-30T06:43:52.133ZDEBGtest_utilization (ExternalAuthn): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = SiloUser { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000000, lookup_type: ById(001de000-5110-4000-8000-000000000000) }, key: 001de000-05e4-4000-8000-000000004007, lookup_type: ById(001de000-05e4-4000-8000-000000004007) } result = Ok(())
31972025-04-30T06:43:52.133ZTRACtest_utilization (ExternalAuthn): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database
31982025-04-30T06:43:52.133ZDEBGtest_utilization (ExternalAuthn): roles roles = RoleSet { roles: {} }
31992025-04-30T06:43:52.134ZDEBGtest_utilization (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
32002025-04-30T06:43:52.136ZTRACtest_utilization (ExternalAuthn): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000000, lookup_type: ById(001de000-5110-4000-8000-000000000000) }
32012025-04-30T06:43:52.136ZTRACtest_utilization (ExternalAuthn): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. } resource_id = 001de000-5110-4000-8000-000000000000 resource_type = Silo
32022025-04-30T06:43:52.136ZTRACtest_utilization (ExternalAuthn): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database
32032025-04-30T06:43:52.136ZDEBGtest_utilization (ExternalAuthn): roles roles = RoleSet { roles: {} }
32042025-04-30T06:43:52.136ZDEBGtest_utilization (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
32052025-04-30T06:43:52.139ZTRACtest_utilization (ExternalAuthn): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
32062025-04-30T06:43:52.139ZTRACtest_utilization (ExternalAuthn): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database
32072025-04-30T06:43:52.139ZDEBGtest_utilization (ExternalAuthn): roles roles = RoleSet { roles: {} }
32082025-04-30T06:43:52.143ZDEBGtest_utilization (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
32092025-04-30T06:43:52.150ZDEBGtest_utilization (ExternalAuthn): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "external-authenticator")} }
32102025-04-30T06:43:52.153ZDEBGtest_utilization (ExternalAuthn): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000000, lookup_type: ById(001de000-5110-4000-8000-000000000000) } result = Ok(())
32112025-04-30T06:43:52.153ZTRACtest_utilization (dropshot_external): authn result: Ok(Context { kind: Authenticated(Details { actor: Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. } }, Some(SiloAuthnPolicy { mapped_fleet_roles: {} })), schemes_tried: [SchemeName("spoof")] }) local_addr = 127.0.0.1:46348 method = POST remote_addr = 127.0.0.1:41283 req_id = dedd9d24-ec89-45d6-bdbf-e4c3b86a030c uri = /v1/system/ip-pools/default/ranges/add
32122025-04-30T06:43:52.153ZTRACtest_utilization (dropshot_external): authorize begin action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true local_addr = 127.0.0.1:46348 method = POST remote_addr = 127.0.0.1:41283 req_id = dedd9d24-ec89-45d6-bdbf-e4c3b86a030c resource = Database uri = /v1/system/ip-pools/default/ranges/add
32132025-04-30T06:43:52.153ZDEBGtest_utilization (dropshot_external): roles actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true local_addr = 127.0.0.1:46348 method = POST remote_addr = 127.0.0.1:41283 req_id = dedd9d24-ec89-45d6-bdbf-e4c3b86a030c roles = RoleSet { roles: {} } uri = /v1/system/ip-pools/default/ranges/add
32142025-04-30T06:43:52.154ZDEBGtest_utilization (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true local_addr = 127.0.0.1:46348 method = POST remote_addr = 127.0.0.1:41283 req_id = dedd9d24-ec89-45d6-bdbf-e4c3b86a030c resource = Database result = Ok(()) uri = /v1/system/ip-pools/default/ranges/add
32152025-04-30T06:43:52.157ZTRACtest_utilization (dropshot_external): authorize begin action = Modify actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true local_addr = 127.0.0.1:46348 method = POST remote_addr = 127.0.0.1:41283 req_id = dedd9d24-ec89-45d6-bdbf-e4c3b86a030c resource = IpPool { parent: Fleet, key: f44520c0-7614-4d06-aa80-97ed717f46ca, lookup_type: ByName("default") } uri = /v1/system/ip-pools/default/ranges/add
32162025-04-30T06:43:52.158ZTRACtest_utilization (dropshot_external): loading roles actor = Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. } actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true local_addr = 127.0.0.1:46348 method = POST remote_addr = 127.0.0.1:41283 req_id = dedd9d24-ec89-45d6-bdbf-e4c3b86a030c resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet uri = /v1/system/ip-pools/default/ranges/add
32172025-04-30T06:43:52.158ZTRACtest_utilization (dropshot_external): authorize begin action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true local_addr = 127.0.0.1:46348 method = POST remote_addr = 127.0.0.1:41283 req_id = dedd9d24-ec89-45d6-bdbf-e4c3b86a030c resource = Database uri = /v1/system/ip-pools/default/ranges/add
32182025-04-30T06:43:52.158ZDEBGtest_utilization (dropshot_external): roles actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true local_addr = 127.0.0.1:46348 method = POST remote_addr = 127.0.0.1:41283 req_id = dedd9d24-ec89-45d6-bdbf-e4c3b86a030c roles = RoleSet { roles: {} } uri = /v1/system/ip-pools/default/ranges/add
32192025-04-30T06:43:52.158ZDEBGtest_utilization (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true local_addr = 127.0.0.1:46348 method = POST remote_addr = 127.0.0.1:41283 req_id = dedd9d24-ec89-45d6-bdbf-e4c3b86a030c resource = Database result = Ok(()) uri = /v1/system/ip-pools/default/ranges/add
32202025-04-30T06:43:52.164ZDEBGtest_utilization (dropshot_external): roles actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true local_addr = 127.0.0.1:46348 method = POST remote_addr = 127.0.0.1:41283 req_id = dedd9d24-ec89-45d6-bdbf-e4c3b86a030c roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} } uri = /v1/system/ip-pools/default/ranges/add
32212025-04-30T06:43:52.168ZDEBGtest_utilization (dropshot_external): authorize result action = Modify actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true local_addr = 127.0.0.1:46348 method = POST remote_addr = 127.0.0.1:41283 req_id = dedd9d24-ec89-45d6-bdbf-e4c3b86a030c resource = IpPool { parent: Fleet, key: f44520c0-7614-4d06-aa80-97ed717f46ca, lookup_type: ByName("default") } result = Ok(()) uri = /v1/system/ip-pools/default/ranges/add
32222025-04-30T06:43:52.169ZTRACtest_utilization (dropshot_external): authorize begin action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true local_addr = 127.0.0.1:46348 method = POST remote_addr = 127.0.0.1:41283 req_id = dedd9d24-ec89-45d6-bdbf-e4c3b86a030c resource = Database uri = /v1/system/ip-pools/default/ranges/add
32232025-04-30T06:43:52.169ZDEBGtest_utilization (dropshot_external): roles actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true local_addr = 127.0.0.1:46348 method = POST remote_addr = 127.0.0.1:41283 req_id = dedd9d24-ec89-45d6-bdbf-e4c3b86a030c roles = RoleSet { roles: {} } uri = /v1/system/ip-pools/default/ranges/add
32242025-04-30T06:43:52.170ZDEBGtest_utilization (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true local_addr = 127.0.0.1:46348 method = POST remote_addr = 127.0.0.1:41283 req_id = dedd9d24-ec89-45d6-bdbf-e4c3b86a030c resource = Database result = Ok(()) uri = /v1/system/ip-pools/default/ranges/add
32252025-04-30T06:43:52.172ZTRACtest_utilization (dropshot_external): authorize begin action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true local_addr = 127.0.0.1:46348 method = POST remote_addr = 127.0.0.1:41283 req_id = dedd9d24-ec89-45d6-bdbf-e4c3b86a030c resource = Database uri = /v1/system/ip-pools/default/ranges/add
32262025-04-30T06:43:52.172ZDEBGtest_utilization (dropshot_external): roles actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true local_addr = 127.0.0.1:46348 method = POST remote_addr = 127.0.0.1:41283 req_id = dedd9d24-ec89-45d6-bdbf-e4c3b86a030c roles = RoleSet { roles: {} } uri = /v1/system/ip-pools/default/ranges/add
32272025-04-30T06:43:52.173ZDEBGtest_utilization (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true local_addr = 127.0.0.1:46348 method = POST remote_addr = 127.0.0.1:41283 req_id = dedd9d24-ec89-45d6-bdbf-e4c3b86a030c resource = Database result = Ok(()) uri = /v1/system/ip-pools/default/ranges/add
32282025-04-30T06:43:52.175ZTRACtest_utilization (dropshot_external): authorize begin action = CreateChild actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true local_addr = 127.0.0.1:46348 method = POST remote_addr = 127.0.0.1:41283 req_id = dedd9d24-ec89-45d6-bdbf-e4c3b86a030c resource = IpPool { parent: Fleet, key: f44520c0-7614-4d06-aa80-97ed717f46ca, lookup_type: ByName("default") } uri = /v1/system/ip-pools/default/ranges/add
32292025-04-30T06:43:52.175ZTRACtest_utilization (dropshot_external): loading roles actor = Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. } actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true local_addr = 127.0.0.1:46348 method = POST remote_addr = 127.0.0.1:41283 req_id = dedd9d24-ec89-45d6-bdbf-e4c3b86a030c resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet uri = /v1/system/ip-pools/default/ranges/add
32302025-04-30T06:43:52.175ZTRACtest_utilization (dropshot_external): authorize begin action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true local_addr = 127.0.0.1:46348 method = POST remote_addr = 127.0.0.1:41283 req_id = dedd9d24-ec89-45d6-bdbf-e4c3b86a030c resource = Database uri = /v1/system/ip-pools/default/ranges/add
32312025-04-30T06:43:52.175ZDEBGtest_utilization (dropshot_external): roles actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true local_addr = 127.0.0.1:46348 method = POST remote_addr = 127.0.0.1:41283 req_id = dedd9d24-ec89-45d6-bdbf-e4c3b86a030c roles = RoleSet { roles: {} } uri = /v1/system/ip-pools/default/ranges/add
32322025-04-30T06:43:52.176ZDEBGtest_utilization (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true local_addr = 127.0.0.1:46348 method = POST remote_addr = 127.0.0.1:41283 req_id = dedd9d24-ec89-45d6-bdbf-e4c3b86a030c resource = Database result = Ok(()) uri = /v1/system/ip-pools/default/ranges/add
32332025-04-30T06:43:52.180ZDEBGtest_utilization (dropshot_external): roles actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true local_addr = 127.0.0.1:46348 method = POST remote_addr = 127.0.0.1:41283 req_id = dedd9d24-ec89-45d6-bdbf-e4c3b86a030c roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} } uri = /v1/system/ip-pools/default/ranges/add
32342025-04-30T06:43:52.182ZDEBGtest_utilization (dropshot_external): authorize result action = CreateChild actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true local_addr = 127.0.0.1:46348 method = POST remote_addr = 127.0.0.1:41283 req_id = dedd9d24-ec89-45d6-bdbf-e4c3b86a030c resource = IpPool { parent: Fleet, key: f44520c0-7614-4d06-aa80-97ed717f46ca, lookup_type: ByName("default") } result = Ok(()) uri = /v1/system/ip-pools/default/ranges/add
32352025-04-30T06:43:52.189ZINFOtest_utilization (dropshot_external): request completed latency_us = 82402 local_addr = 127.0.0.1:46348 method = POST remote_addr = 127.0.0.1:41283 req_id = dedd9d24-ec89-45d6-bdbf-e4c3b86a030c response_code = 201 uri = /v1/system/ip-pools/default/ranges/add
32362025-04-30T06:43:52.190ZINFOtest_utilization (external client test context): client received response status = 201
32372025-04-30T06:43:52.190ZINFOtest_utilization (external client test context): client request body = Body { inner: BoxBody } method = POST uri = http://127.0.0.1:46348/v1/system/ip-pools/default/silos
32382025-04-30T06:43:52.191ZTRACtest_utilization (dropshot_external): incoming request local_addr = 127.0.0.1:46348 method = POST remote_addr = 127.0.0.1:41283 req_id = 630c489d-9f33-4c6a-96c2-a55345b633c8 uri = /v1/system/ip-pools/default/silos
32392025-04-30T06:43:52.191ZTRACtest_utilization (dropshot_external): authn: trying SchemeName("spoof") local_addr = 127.0.0.1:46348 method = POST remote_addr = 127.0.0.1:41283 req_id = 630c489d-9f33-4c6a-96c2-a55345b633c8 uri = /v1/system/ip-pools/default/silos
32402025-04-30T06:43:52.191ZDEBGtest_utilization (dropshot_external): looking up silo for user local_addr = 127.0.0.1:46348 method = POST remote_addr = 127.0.0.1:41283 req_id = 630c489d-9f33-4c6a-96c2-a55345b633c8 silo_user_id = 001de000-05e4-4000-8000-000000004007 uri = /v1/system/ip-pools/default/silos
32412025-04-30T06:43:52.191ZTRACtest_utilization (ExternalAuthn): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database
32422025-04-30T06:43:52.191ZDEBGtest_utilization (ExternalAuthn): roles roles = RoleSet { roles: {} }
32432025-04-30T06:43:52.192ZDEBGtest_utilization (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
32442025-04-30T06:43:52.194ZTRACtest_utilization (ExternalAuthn): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database
32452025-04-30T06:43:52.195ZDEBGtest_utilization (ExternalAuthn): roles roles = RoleSet { roles: {} }
32462025-04-30T06:43:52.195ZDEBGtest_utilization (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
32472025-04-30T06:43:52.197ZTRACtest_utilization (ExternalAuthn): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = SiloUser { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000000, lookup_type: ById(001de000-5110-4000-8000-000000000000) }, key: 001de000-05e4-4000-8000-000000004007, lookup_type: ById(001de000-05e4-4000-8000-000000004007) }
32482025-04-30T06:43:52.197ZTRACtest_utilization (ExternalAuthn): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. } resource_id = 001de000-5110-4000-8000-000000000000 resource_type = Silo
32492025-04-30T06:43:52.198ZTRACtest_utilization (ExternalAuthn): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database
32502025-04-30T06:43:52.198ZDEBGtest_utilization (ExternalAuthn): roles roles = RoleSet { roles: {} }
32512025-04-30T06:43:52.198ZDEBGtest_utilization (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
32522025-04-30T06:43:52.200ZTRACtest_utilization (ExternalAuthn): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
32532025-04-30T06:43:52.200ZTRACtest_utilization (ExternalAuthn): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database
32542025-04-30T06:43:52.200ZDEBGtest_utilization (ExternalAuthn): roles roles = RoleSet { roles: {} }
32552025-04-30T06:43:52.201ZDEBGtest_utilization (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
32562025-04-30T06:43:52.211ZDEBGtest_utilization (ExternalAuthn): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "external-authenticator")} }
32572025-04-30T06:43:52.218ZDEBGtest_utilization (ExternalAuthn): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = SiloUser { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000000, lookup_type: ById(001de000-5110-4000-8000-000000000000) }, key: 001de000-05e4-4000-8000-000000004007, lookup_type: ById(001de000-05e4-4000-8000-000000004007) } result = Ok(())
32582025-04-30T06:43:52.219ZTRACtest_utilization (ExternalAuthn): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database
32592025-04-30T06:43:52.219ZDEBGtest_utilization (ExternalAuthn): roles roles = RoleSet { roles: {} }
32602025-04-30T06:43:52.220ZDEBGtest_utilization (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
32612025-04-30T06:43:52.224ZTRACtest_utilization (ExternalAuthn): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000000, lookup_type: ById(001de000-5110-4000-8000-000000000000) }
32622025-04-30T06:43:52.224ZTRACtest_utilization (ExternalAuthn): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. } resource_id = 001de000-5110-4000-8000-000000000000 resource_type = Silo
32632025-04-30T06:43:52.224ZTRACtest_utilization (ExternalAuthn): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database
32642025-04-30T06:43:52.224ZDEBGtest_utilization (ExternalAuthn): roles roles = RoleSet { roles: {} }
32652025-04-30T06:43:52.225ZDEBGtest_utilization (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
32662025-04-30T06:43:52.227ZTRACtest_utilization (ExternalAuthn): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
32672025-04-30T06:43:52.227ZTRACtest_utilization (ExternalAuthn): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database
32682025-04-30T06:43:52.227ZDEBGtest_utilization (ExternalAuthn): roles roles = RoleSet { roles: {} }
32692025-04-30T06:43:52.228ZDEBGtest_utilization (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
32702025-04-30T06:43:52.231ZDEBGtest_utilization (ExternalAuthn): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "external-authenticator")} }
32712025-04-30T06:43:52.237ZDEBGtest_utilization (ExternalAuthn): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000000, lookup_type: ById(001de000-5110-4000-8000-000000000000) } result = Ok(())
32722025-04-30T06:43:52.237ZTRACtest_utilization (dropshot_external): authn result: Ok(Context { kind: Authenticated(Details { actor: Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. } }, Some(SiloAuthnPolicy { mapped_fleet_roles: {} })), schemes_tried: [SchemeName("spoof")] }) local_addr = 127.0.0.1:46348 method = POST remote_addr = 127.0.0.1:41283 req_id = 630c489d-9f33-4c6a-96c2-a55345b633c8 uri = /v1/system/ip-pools/default/silos
32732025-04-30T06:43:52.237ZTRACtest_utilization (dropshot_external): authorize begin action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true local_addr = 127.0.0.1:46348 method = POST remote_addr = 127.0.0.1:41283 req_id = 630c489d-9f33-4c6a-96c2-a55345b633c8 resource = Database uri = /v1/system/ip-pools/default/silos
32742025-04-30T06:43:52.238ZDEBGtest_utilization (dropshot_external): roles actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true local_addr = 127.0.0.1:46348 method = POST remote_addr = 127.0.0.1:41283 req_id = 630c489d-9f33-4c6a-96c2-a55345b633c8 roles = RoleSet { roles: {} } uri = /v1/system/ip-pools/default/silos
32752025-04-30T06:43:52.238ZDEBGtest_utilization (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true local_addr = 127.0.0.1:46348 method = POST remote_addr = 127.0.0.1:41283 req_id = 630c489d-9f33-4c6a-96c2-a55345b633c8 resource = Database result = Ok(()) uri = /v1/system/ip-pools/default/silos
32762025-04-30T06:43:52.258ZTRACtest_utilization (dropshot_external): authorize begin action = Modify actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true local_addr = 127.0.0.1:46348 method = POST remote_addr = 127.0.0.1:41283 req_id = 630c489d-9f33-4c6a-96c2-a55345b633c8 resource = IpPool { parent: Fleet, key: f44520c0-7614-4d06-aa80-97ed717f46ca, lookup_type: ByName("default") } uri = /v1/system/ip-pools/default/silos
32772025-04-30T06:43:52.258ZTRACtest_utilization (dropshot_external): loading roles actor = Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. } actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true local_addr = 127.0.0.1:46348 method = POST remote_addr = 127.0.0.1:41283 req_id = 630c489d-9f33-4c6a-96c2-a55345b633c8 resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet uri = /v1/system/ip-pools/default/silos
32782025-04-30T06:43:52.258ZTRACtest_utilization (dropshot_external): authorize begin action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true local_addr = 127.0.0.1:46348 method = POST remote_addr = 127.0.0.1:41283 req_id = 630c489d-9f33-4c6a-96c2-a55345b633c8 resource = Database uri = /v1/system/ip-pools/default/silos
32792025-04-30T06:43:52.258ZDEBGtest_utilization (dropshot_external): roles actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true local_addr = 127.0.0.1:46348 method = POST remote_addr = 127.0.0.1:41283 req_id = 630c489d-9f33-4c6a-96c2-a55345b633c8 roles = RoleSet { roles: {} } uri = /v1/system/ip-pools/default/silos
32802025-04-30T06:43:52.259ZDEBGtest_utilization (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true local_addr = 127.0.0.1:46348 method = POST remote_addr = 127.0.0.1:41283 req_id = 630c489d-9f33-4c6a-96c2-a55345b633c8 resource = Database result = Ok(()) uri = /v1/system/ip-pools/default/silos
32812025-04-30T06:43:52.267ZDEBGtest_utilization (dropshot_external): roles actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true local_addr = 127.0.0.1:46348 method = POST remote_addr = 127.0.0.1:41283 req_id = 630c489d-9f33-4c6a-96c2-a55345b633c8 roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} } uri = /v1/system/ip-pools/default/silos
32822025-04-30T06:43:52.273ZDEBGtest_utilization (dropshot_external): authorize result action = Modify actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true local_addr = 127.0.0.1:46348 method = POST remote_addr = 127.0.0.1:41283 req_id = 630c489d-9f33-4c6a-96c2-a55345b633c8 resource = IpPool { parent: Fleet, key: f44520c0-7614-4d06-aa80-97ed717f46ca, lookup_type: ByName("default") } result = Ok(()) uri = /v1/system/ip-pools/default/silos
32832025-04-30T06:43:52.273ZTRACtest_utilization (dropshot_external): authorize begin action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true local_addr = 127.0.0.1:46348 method = POST remote_addr = 127.0.0.1:41283 req_id = 630c489d-9f33-4c6a-96c2-a55345b633c8 resource = Database uri = /v1/system/ip-pools/default/silos
32842025-04-30T06:43:52.273ZDEBGtest_utilization (dropshot_external): roles actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true local_addr = 127.0.0.1:46348 method = POST remote_addr = 127.0.0.1:41283 req_id = 630c489d-9f33-4c6a-96c2-a55345b633c8 roles = RoleSet { roles: {} } uri = /v1/system/ip-pools/default/silos
32852025-04-30T06:43:52.274ZDEBGtest_utilization (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true local_addr = 127.0.0.1:46348 method = POST remote_addr = 127.0.0.1:41283 req_id = 630c489d-9f33-4c6a-96c2-a55345b633c8 resource = Database result = Ok(()) uri = /v1/system/ip-pools/default/silos
32862025-04-30T06:43:52.276ZTRACtest_utilization (dropshot_external): authorize begin action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true local_addr = 127.0.0.1:46348 method = POST remote_addr = 127.0.0.1:41283 req_id = 630c489d-9f33-4c6a-96c2-a55345b633c8 resource = Database uri = /v1/system/ip-pools/default/silos
32872025-04-30T06:43:52.276ZDEBGtest_utilization (dropshot_external): roles actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true local_addr = 127.0.0.1:46348 method = POST remote_addr = 127.0.0.1:41283 req_id = 630c489d-9f33-4c6a-96c2-a55345b633c8 roles = RoleSet { roles: {} } uri = /v1/system/ip-pools/default/silos
32882025-04-30T06:43:52.277ZDEBGtest_utilization (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true local_addr = 127.0.0.1:46348 method = POST remote_addr = 127.0.0.1:41283 req_id = 630c489d-9f33-4c6a-96c2-a55345b633c8 resource = Database result = Ok(()) uri = /v1/system/ip-pools/default/silos
32892025-04-30T06:43:52.279ZTRACtest_utilization (dropshot_external): authorize begin action = Modify actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true local_addr = 127.0.0.1:46348 method = POST remote_addr = 127.0.0.1:41283 req_id = 630c489d-9f33-4c6a-96c2-a55345b633c8 resource = Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000000, lookup_type: ById(001de000-5110-4000-8000-000000000000) } uri = /v1/system/ip-pools/default/silos
32902025-04-30T06:43:52.279ZTRACtest_utilization (dropshot_external): loading roles actor = Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. } actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true local_addr = 127.0.0.1:46348 method = POST remote_addr = 127.0.0.1:41283 req_id = 630c489d-9f33-4c6a-96c2-a55345b633c8 resource_id = 001de000-5110-4000-8000-000000000000 resource_type = Silo uri = /v1/system/ip-pools/default/silos
32912025-04-30T06:43:52.280ZTRACtest_utilization (dropshot_external): authorize begin action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true local_addr = 127.0.0.1:46348 method = POST remote_addr = 127.0.0.1:41283 req_id = 630c489d-9f33-4c6a-96c2-a55345b633c8 resource = Database uri = /v1/system/ip-pools/default/silos
32922025-04-30T06:43:52.280ZDEBGtest_utilization (dropshot_external): roles actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true local_addr = 127.0.0.1:46348 method = POST remote_addr = 127.0.0.1:41283 req_id = 630c489d-9f33-4c6a-96c2-a55345b633c8 roles = RoleSet { roles: {} } uri = /v1/system/ip-pools/default/silos
32932025-04-30T06:43:52.281ZDEBGtest_utilization (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true local_addr = 127.0.0.1:46348 method = POST remote_addr = 127.0.0.1:41283 req_id = 630c489d-9f33-4c6a-96c2-a55345b633c8 resource = Database result = Ok(()) uri = /v1/system/ip-pools/default/silos
32942025-04-30T06:43:52.285ZTRACtest_utilization (dropshot_external): loading roles actor = Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. } actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true local_addr = 127.0.0.1:46348 method = POST remote_addr = 127.0.0.1:41283 req_id = 630c489d-9f33-4c6a-96c2-a55345b633c8 resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet uri = /v1/system/ip-pools/default/silos
32952025-04-30T06:43:52.285ZTRACtest_utilization (dropshot_external): authorize begin action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true local_addr = 127.0.0.1:46348 method = POST remote_addr = 127.0.0.1:41283 req_id = 630c489d-9f33-4c6a-96c2-a55345b633c8 resource = Database uri = /v1/system/ip-pools/default/silos
32962025-04-30T06:43:52.285ZDEBGtest_utilization (dropshot_external): roles actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true local_addr = 127.0.0.1:46348 method = POST remote_addr = 127.0.0.1:41283 req_id = 630c489d-9f33-4c6a-96c2-a55345b633c8 roles = RoleSet { roles: {} } uri = /v1/system/ip-pools/default/silos
32972025-04-30T06:43:52.286ZDEBGtest_utilization (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true local_addr = 127.0.0.1:46348 method = POST remote_addr = 127.0.0.1:41283 req_id = 630c489d-9f33-4c6a-96c2-a55345b633c8 resource = Database result = Ok(()) uri = /v1/system/ip-pools/default/silos
32982025-04-30T06:43:52.289ZDEBGtest_utilization (dropshot_external): roles actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true local_addr = 127.0.0.1:46348 method = POST remote_addr = 127.0.0.1:41283 req_id = 630c489d-9f33-4c6a-96c2-a55345b633c8 roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin"), (Silo, 001de000-5110-4000-8000-000000000000, "admin")} } uri = /v1/system/ip-pools/default/silos
32992025-04-30T06:43:52.292ZDEBGtest_utilization (dropshot_external): authorize result action = Modify actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true local_addr = 127.0.0.1:46348 method = POST remote_addr = 127.0.0.1:41283 req_id = 630c489d-9f33-4c6a-96c2-a55345b633c8 resource = Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000000, lookup_type: ById(001de000-5110-4000-8000-000000000000) } result = Ok(()) uri = /v1/system/ip-pools/default/silos
33002025-04-30T06:43:52.292ZTRACtest_utilization (dropshot_external): authorize begin action = CreateChild actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true local_addr = 127.0.0.1:46348 method = POST remote_addr = 127.0.0.1:41283 req_id = 630c489d-9f33-4c6a-96c2-a55345b633c8 resource = IpPoolList uri = /v1/system/ip-pools/default/silos
33012025-04-30T06:43:52.292ZTRACtest_utilization (dropshot_external): loading roles actor = Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. } actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true local_addr = 127.0.0.1:46348 method = POST remote_addr = 127.0.0.1:41283 req_id = 630c489d-9f33-4c6a-96c2-a55345b633c8 resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet uri = /v1/system/ip-pools/default/silos
33022025-04-30T06:43:52.292ZTRACtest_utilization (dropshot_external): authorize begin action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true local_addr = 127.0.0.1:46348 method = POST remote_addr = 127.0.0.1:41283 req_id = 630c489d-9f33-4c6a-96c2-a55345b633c8 resource = Database uri = /v1/system/ip-pools/default/silos
33032025-04-30T06:43:52.292ZDEBGtest_utilization (dropshot_external): roles actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true local_addr = 127.0.0.1:46348 method = POST remote_addr = 127.0.0.1:41283 req_id = 630c489d-9f33-4c6a-96c2-a55345b633c8 roles = RoleSet { roles: {} } uri = /v1/system/ip-pools/default/silos
33042025-04-30T06:43:52.292ZDEBGtest_utilization (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true local_addr = 127.0.0.1:46348 method = POST remote_addr = 127.0.0.1:41283 req_id = 630c489d-9f33-4c6a-96c2-a55345b633c8 resource = Database result = Ok(()) uri = /v1/system/ip-pools/default/silos
33052025-04-30T06:43:52.297ZDEBGtest_utilization (dropshot_external): roles actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true local_addr = 127.0.0.1:46348 method = POST remote_addr = 127.0.0.1:41283 req_id = 630c489d-9f33-4c6a-96c2-a55345b633c8 roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} } uri = /v1/system/ip-pools/default/silos
33062025-04-30T06:43:52.301ZDEBGtest_utilization (dropshot_external): authorize result action = CreateChild actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true local_addr = 127.0.0.1:46348 method = POST remote_addr = 127.0.0.1:41283 req_id = 630c489d-9f33-4c6a-96c2-a55345b633c8 resource = IpPoolList result = Ok(()) uri = /v1/system/ip-pools/default/silos
33072025-04-30T06:43:52.301ZTRACtest_utilization (dropshot_external): authorize begin action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true local_addr = 127.0.0.1:46348 method = POST remote_addr = 127.0.0.1:41283 req_id = 630c489d-9f33-4c6a-96c2-a55345b633c8 resource = Database uri = /v1/system/ip-pools/default/silos
33082025-04-30T06:43:52.301ZDEBGtest_utilization (dropshot_external): roles actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true local_addr = 127.0.0.1:46348 method = POST remote_addr = 127.0.0.1:41283 req_id = 630c489d-9f33-4c6a-96c2-a55345b633c8 roles = RoleSet { roles: {} } uri = /v1/system/ip-pools/default/silos
33092025-04-30T06:43:52.302ZDEBGtest_utilization (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true local_addr = 127.0.0.1:46348 method = POST remote_addr = 127.0.0.1:41283 req_id = 630c489d-9f33-4c6a-96c2-a55345b633c8 resource = Database result = Ok(()) uri = /v1/system/ip-pools/default/silos
33102025-04-30T06:43:52.357ZTRACtest_utilization (results-sink): batch interval expired, but no samples to insert collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
33112025-04-30T06:43:52.357ZTRACtest_utilization (results-sink-cluster): batch interval expired, but no samples to insert collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
33122025-04-30T06:43:52.362ZINFOtest_utilization (dropshot_external): request completed latency_us = 171353 local_addr = 127.0.0.1:46348 method = POST remote_addr = 127.0.0.1:41283 req_id = 630c489d-9f33-4c6a-96c2-a55345b633c8 response_code = 201 uri = /v1/system/ip-pools/default/silos
33132025-04-30T06:43:52.363ZINFOtest_utilization (external client test context): client received response status = 201
33142025-04-30T06:43:52.363ZINFOtest_utilization (external client test context): client request body = Body { inner: BoxBody } method = PUT uri = http://127.0.0.1:46348/v1/system/silos/test-suite-silo/quotas
33152025-04-30T06:43:52.364ZTRACtest_utilization (dropshot_external): incoming request local_addr = 127.0.0.1:46348 method = PUT remote_addr = 127.0.0.1:41283 req_id = 957d1649-9843-4451-b116-7570fe502cc1 uri = /v1/system/silos/test-suite-silo/quotas
33162025-04-30T06:43:52.364ZTRACtest_utilization (dropshot_external): authn: trying SchemeName("spoof") local_addr = 127.0.0.1:46348 method = PUT remote_addr = 127.0.0.1:41283 req_id = 957d1649-9843-4451-b116-7570fe502cc1 uri = /v1/system/silos/test-suite-silo/quotas
33172025-04-30T06:43:52.364ZDEBGtest_utilization (dropshot_external): looking up silo for user local_addr = 127.0.0.1:46348 method = PUT remote_addr = 127.0.0.1:41283 req_id = 957d1649-9843-4451-b116-7570fe502cc1 silo_user_id = 001de000-05e4-4000-8000-000000004007 uri = /v1/system/silos/test-suite-silo/quotas
33182025-04-30T06:43:52.364ZTRACtest_utilization (ExternalAuthn): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database
33192025-04-30T06:43:52.365ZDEBGtest_utilization (ExternalAuthn): roles roles = RoleSet { roles: {} }
33202025-04-30T06:43:52.365ZDEBGtest_utilization (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
33212025-04-30T06:43:52.369ZTRACtest_utilization (ExternalAuthn): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database
33222025-04-30T06:43:52.369ZDEBGtest_utilization (ExternalAuthn): roles roles = RoleSet { roles: {} }
33232025-04-30T06:43:52.370ZDEBGtest_utilization (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
33242025-04-30T06:43:52.372ZTRACtest_utilization (ExternalAuthn): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = SiloUser { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000000, lookup_type: ById(001de000-5110-4000-8000-000000000000) }, key: 001de000-05e4-4000-8000-000000004007, lookup_type: ById(001de000-05e4-4000-8000-000000004007) }
33252025-04-30T06:43:52.372ZTRACtest_utilization (ExternalAuthn): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. } resource_id = 001de000-5110-4000-8000-000000000000 resource_type = Silo
33262025-04-30T06:43:52.372ZTRACtest_utilization (ExternalAuthn): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database
33272025-04-30T06:43:52.372ZDEBGtest_utilization (ExternalAuthn): roles roles = RoleSet { roles: {} }
33282025-04-30T06:43:52.373ZDEBGtest_utilization (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
33292025-04-30T06:43:52.375ZTRACtest_utilization (ExternalAuthn): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
33302025-04-30T06:43:52.375ZTRACtest_utilization (ExternalAuthn): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database
33312025-04-30T06:43:52.375ZDEBGtest_utilization (ExternalAuthn): roles roles = RoleSet { roles: {} }
33322025-04-30T06:43:52.376ZDEBGtest_utilization (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
33332025-04-30T06:43:52.378ZDEBGtest_utilization (ExternalAuthn): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "external-authenticator")} }
33342025-04-30T06:43:52.383ZDEBGtest_utilization (ExternalAuthn): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = SiloUser { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000000, lookup_type: ById(001de000-5110-4000-8000-000000000000) }, key: 001de000-05e4-4000-8000-000000004007, lookup_type: ById(001de000-05e4-4000-8000-000000004007) } result = Ok(())
33352025-04-30T06:43:52.383ZTRACtest_utilization (ExternalAuthn): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database
33362025-04-30T06:43:52.383ZDEBGtest_utilization (ExternalAuthn): roles roles = RoleSet { roles: {} }
33372025-04-30T06:43:52.384ZDEBGtest_utilization (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
33382025-04-30T06:43:52.386ZTRACtest_utilization (ExternalAuthn): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000000, lookup_type: ById(001de000-5110-4000-8000-000000000000) }
33392025-04-30T06:43:52.386ZTRACtest_utilization (ExternalAuthn): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. } resource_id = 001de000-5110-4000-8000-000000000000 resource_type = Silo
33402025-04-30T06:43:52.386ZTRACtest_utilization (ExternalAuthn): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database
33412025-04-30T06:43:52.386ZDEBGtest_utilization (ExternalAuthn): roles roles = RoleSet { roles: {} }
33422025-04-30T06:43:52.387ZDEBGtest_utilization (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
33432025-04-30T06:43:52.389ZTRACtest_utilization (ExternalAuthn): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
33442025-04-30T06:43:52.389ZTRACtest_utilization (ExternalAuthn): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database
33452025-04-30T06:43:52.389ZDEBGtest_utilization (ExternalAuthn): roles roles = RoleSet { roles: {} }
33462025-04-30T06:43:52.390ZDEBGtest_utilization (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
33472025-04-30T06:43:52.392ZDEBGtest_utilization (ExternalAuthn): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "external-authenticator")} }
33482025-04-30T06:43:52.395ZDEBGtest_utilization (ExternalAuthn): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000000, lookup_type: ById(001de000-5110-4000-8000-000000000000) } result = Ok(())
33492025-04-30T06:43:52.395ZTRACtest_utilization (dropshot_external): authn result: Ok(Context { kind: Authenticated(Details { actor: Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. } }, Some(SiloAuthnPolicy { mapped_fleet_roles: {} })), schemes_tried: [SchemeName("spoof")] }) local_addr = 127.0.0.1:46348 method = PUT remote_addr = 127.0.0.1:41283 req_id = 957d1649-9843-4451-b116-7570fe502cc1 uri = /v1/system/silos/test-suite-silo/quotas
33502025-04-30T06:43:52.395ZTRACtest_utilization (dropshot_external): authorize begin action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true local_addr = 127.0.0.1:46348 method = PUT remote_addr = 127.0.0.1:41283 req_id = 957d1649-9843-4451-b116-7570fe502cc1 resource = Database uri = /v1/system/silos/test-suite-silo/quotas
33512025-04-30T06:43:52.395ZDEBGtest_utilization (dropshot_external): roles actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true local_addr = 127.0.0.1:46348 method = PUT remote_addr = 127.0.0.1:41283 req_id = 957d1649-9843-4451-b116-7570fe502cc1 roles = RoleSet { roles: {} } uri = /v1/system/silos/test-suite-silo/quotas
33522025-04-30T06:43:52.396ZDEBGtest_utilization (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true local_addr = 127.0.0.1:46348 method = PUT remote_addr = 127.0.0.1:41283 req_id = 957d1649-9843-4451-b116-7570fe502cc1 resource = Database result = Ok(()) uri = /v1/system/silos/test-suite-silo/quotas
33532025-04-30T06:43:52.400ZTRACtest_utilization (dropshot_external): authorize begin action = Modify actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true local_addr = 127.0.0.1:46348 method = PUT remote_addr = 127.0.0.1:41283 req_id = 957d1649-9843-4451-b116-7570fe502cc1 resource = Silo { parent: Fleet, key: ad91dcf9-5eb8-4a8d-b721-b5a47ceac136, lookup_type: ByName("test-suite-silo") } uri = /v1/system/silos/test-suite-silo/quotas
33542025-04-30T06:43:52.400ZTRACtest_utilization (dropshot_external): loading roles actor = Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. } actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true local_addr = 127.0.0.1:46348 method = PUT remote_addr = 127.0.0.1:41283 req_id = 957d1649-9843-4451-b116-7570fe502cc1 resource_id = ad91dcf9-5eb8-4a8d-b721-b5a47ceac136 resource_type = Silo uri = /v1/system/silos/test-suite-silo/quotas
33552025-04-30T06:43:52.400ZTRACtest_utilization (dropshot_external): authorize begin action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true local_addr = 127.0.0.1:46348 method = PUT remote_addr = 127.0.0.1:41283 req_id = 957d1649-9843-4451-b116-7570fe502cc1 resource = Database uri = /v1/system/silos/test-suite-silo/quotas
33562025-04-30T06:43:52.400ZDEBGtest_utilization (dropshot_external): roles actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true local_addr = 127.0.0.1:46348 method = PUT remote_addr = 127.0.0.1:41283 req_id = 957d1649-9843-4451-b116-7570fe502cc1 roles = RoleSet { roles: {} } uri = /v1/system/silos/test-suite-silo/quotas
33572025-04-30T06:43:52.401ZDEBGtest_utilization (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true local_addr = 127.0.0.1:46348 method = PUT remote_addr = 127.0.0.1:41283 req_id = 957d1649-9843-4451-b116-7570fe502cc1 resource = Database result = Ok(()) uri = /v1/system/silos/test-suite-silo/quotas
33582025-04-30T06:43:52.404ZTRACtest_utilization (dropshot_external): loading roles actor = Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. } actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true local_addr = 127.0.0.1:46348 method = PUT remote_addr = 127.0.0.1:41283 req_id = 957d1649-9843-4451-b116-7570fe502cc1 resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet uri = /v1/system/silos/test-suite-silo/quotas
33592025-04-30T06:43:52.404ZTRACtest_utilization (dropshot_external): authorize begin action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true local_addr = 127.0.0.1:46348 method = PUT remote_addr = 127.0.0.1:41283 req_id = 957d1649-9843-4451-b116-7570fe502cc1 resource = Database uri = /v1/system/silos/test-suite-silo/quotas
33602025-04-30T06:43:52.404ZDEBGtest_utilization (dropshot_external): roles actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true local_addr = 127.0.0.1:46348 method = PUT remote_addr = 127.0.0.1:41283 req_id = 957d1649-9843-4451-b116-7570fe502cc1 roles = RoleSet { roles: {} } uri = /v1/system/silos/test-suite-silo/quotas
33612025-04-30T06:43:52.405ZDEBGtest_utilization (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true local_addr = 127.0.0.1:46348 method = PUT remote_addr = 127.0.0.1:41283 req_id = 957d1649-9843-4451-b116-7570fe502cc1 resource = Database result = Ok(()) uri = /v1/system/silos/test-suite-silo/quotas
33622025-04-30T06:43:52.409ZDEBGtest_utilization (dropshot_external): roles actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true local_addr = 127.0.0.1:46348 method = PUT remote_addr = 127.0.0.1:41283 req_id = 957d1649-9843-4451-b116-7570fe502cc1 roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} } uri = /v1/system/silos/test-suite-silo/quotas
33632025-04-30T06:43:52.415ZDEBGtest_utilization (dropshot_external): authorize result action = Modify actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true local_addr = 127.0.0.1:46348 method = PUT remote_addr = 127.0.0.1:41283 req_id = 957d1649-9843-4451-b116-7570fe502cc1 resource = Silo { parent: Fleet, key: ad91dcf9-5eb8-4a8d-b721-b5a47ceac136, lookup_type: ByName("test-suite-silo") } result = Ok(()) uri = /v1/system/silos/test-suite-silo/quotas
33642025-04-30T06:43:52.415ZTRACtest_utilization (dropshot_external): authorize begin action = Modify actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true local_addr = 127.0.0.1:46348 method = PUT remote_addr = 127.0.0.1:41283 req_id = 957d1649-9843-4451-b116-7570fe502cc1 resource = Silo { parent: Fleet, key: ad91dcf9-5eb8-4a8d-b721-b5a47ceac136, lookup_type: ByName("test-suite-silo") } uri = /v1/system/silos/test-suite-silo/quotas
33652025-04-30T06:43:52.415ZTRACtest_utilization (dropshot_external): loading roles actor = Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. } actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true local_addr = 127.0.0.1:46348 method = PUT remote_addr = 127.0.0.1:41283 req_id = 957d1649-9843-4451-b116-7570fe502cc1 resource_id = ad91dcf9-5eb8-4a8d-b721-b5a47ceac136 resource_type = Silo uri = /v1/system/silos/test-suite-silo/quotas
33662025-04-30T06:43:52.415ZTRACtest_utilization (dropshot_external): authorize begin action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true local_addr = 127.0.0.1:46348 method = PUT remote_addr = 127.0.0.1:41283 req_id = 957d1649-9843-4451-b116-7570fe502cc1 resource = Database uri = /v1/system/silos/test-suite-silo/quotas
33672025-04-30T06:43:52.415ZDEBGtest_utilization (dropshot_external): roles actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true local_addr = 127.0.0.1:46348 method = PUT remote_addr = 127.0.0.1:41283 req_id = 957d1649-9843-4451-b116-7570fe502cc1 roles = RoleSet { roles: {} } uri = /v1/system/silos/test-suite-silo/quotas
33682025-04-30T06:43:52.415ZDEBGtest_utilization (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true local_addr = 127.0.0.1:46348 method = PUT remote_addr = 127.0.0.1:41283 req_id = 957d1649-9843-4451-b116-7570fe502cc1 resource = Database result = Ok(()) uri = /v1/system/silos/test-suite-silo/quotas
33692025-04-30T06:43:52.419ZTRACtest_utilization (dropshot_external): loading roles actor = Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. } actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true local_addr = 127.0.0.1:46348 method = PUT remote_addr = 127.0.0.1:41283 req_id = 957d1649-9843-4451-b116-7570fe502cc1 resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet uri = /v1/system/silos/test-suite-silo/quotas
33702025-04-30T06:43:52.419ZTRACtest_utilization (dropshot_external): authorize begin action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true local_addr = 127.0.0.1:46348 method = PUT remote_addr = 127.0.0.1:41283 req_id = 957d1649-9843-4451-b116-7570fe502cc1 resource = Database uri = /v1/system/silos/test-suite-silo/quotas
33712025-04-30T06:43:52.419ZDEBGtest_utilization (dropshot_external): roles actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true local_addr = 127.0.0.1:46348 method = PUT remote_addr = 127.0.0.1:41283 req_id = 957d1649-9843-4451-b116-7570fe502cc1 roles = RoleSet { roles: {} } uri = /v1/system/silos/test-suite-silo/quotas
33722025-04-30T06:43:52.420ZDEBGtest_utilization (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true local_addr = 127.0.0.1:46348 method = PUT remote_addr = 127.0.0.1:41283 req_id = 957d1649-9843-4451-b116-7570fe502cc1 resource = Database result = Ok(()) uri = /v1/system/silos/test-suite-silo/quotas
33732025-04-30T06:43:52.425ZDEBGtest_utilization (dropshot_external): roles actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true local_addr = 127.0.0.1:46348 method = PUT remote_addr = 127.0.0.1:41283 req_id = 957d1649-9843-4451-b116-7570fe502cc1 roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} } uri = /v1/system/silos/test-suite-silo/quotas
33742025-04-30T06:43:52.430ZDEBGtest_utilization (dropshot_external): authorize result action = Modify actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true local_addr = 127.0.0.1:46348 method = PUT remote_addr = 127.0.0.1:41283 req_id = 957d1649-9843-4451-b116-7570fe502cc1 resource = Silo { parent: Fleet, key: ad91dcf9-5eb8-4a8d-b721-b5a47ceac136, lookup_type: ByName("test-suite-silo") } result = Ok(()) uri = /v1/system/silos/test-suite-silo/quotas
33752025-04-30T06:43:52.435ZTRACtest_utilization (dropshot_external): authorize begin action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true local_addr = 127.0.0.1:46348 method = PUT remote_addr = 127.0.0.1:41283 req_id = 957d1649-9843-4451-b116-7570fe502cc1 resource = Database uri = /v1/system/silos/test-suite-silo/quotas
33762025-04-30T06:43:52.435ZDEBGtest_utilization (dropshot_external): roles actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true local_addr = 127.0.0.1:46348 method = PUT remote_addr = 127.0.0.1:41283 req_id = 957d1649-9843-4451-b116-7570fe502cc1 roles = RoleSet { roles: {} } uri = /v1/system/silos/test-suite-silo/quotas
33772025-04-30T06:43:52.436ZDEBGtest_utilization (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true local_addr = 127.0.0.1:46348 method = PUT remote_addr = 127.0.0.1:41283 req_id = 957d1649-9843-4451-b116-7570fe502cc1 resource = Database result = Ok(()) uri = /v1/system/silos/test-suite-silo/quotas
33782025-04-30T06:43:52.442ZINFOtest_utilization (dropshot_external): request completed latency_us = 78011 local_addr = 127.0.0.1:46348 method = PUT remote_addr = 127.0.0.1:41283 req_id = 957d1649-9843-4451-b116-7570fe502cc1 response_code = 200 uri = /v1/system/silos/test-suite-silo/quotas
33792025-04-30T06:43:52.444ZINFOtest_utilization (external client test context): client received response status = 200
33802025-04-30T06:43:52.444ZINFOtest_utilization (external client test context): client request body = Body { inner: BoxBody } method = GET uri = http://127.0.0.1:46348/v1/system/utilization/silos
33812025-04-30T06:43:52.444ZTRACtest_utilization (dropshot_external): incoming request local_addr = 127.0.0.1:46348 method = GET remote_addr = 127.0.0.1:41283 req_id = c8edc401-8413-432d-a9a3-d491c17d9b13 uri = /v1/system/utilization/silos
33822025-04-30T06:43:52.444ZTRACtest_utilization (dropshot_external): authn: trying SchemeName("spoof") local_addr = 127.0.0.1:46348 method = GET remote_addr = 127.0.0.1:41283 req_id = c8edc401-8413-432d-a9a3-d491c17d9b13 uri = /v1/system/utilization/silos
33832025-04-30T06:43:52.444ZDEBGtest_utilization (dropshot_external): looking up silo for user local_addr = 127.0.0.1:46348 method = GET remote_addr = 127.0.0.1:41283 req_id = c8edc401-8413-432d-a9a3-d491c17d9b13 silo_user_id = 001de000-05e4-4000-8000-000000004007 uri = /v1/system/utilization/silos
33842025-04-30T06:43:52.445ZTRACtest_utilization (ExternalAuthn): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database
33852025-04-30T06:43:52.445ZDEBGtest_utilization (ExternalAuthn): roles roles = RoleSet { roles: {} }
33862025-04-30T06:43:52.445ZDEBGtest_utilization (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
33872025-04-30T06:43:52.449ZTRACtest_utilization (ExternalAuthn): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database
33882025-04-30T06:43:52.449ZDEBGtest_utilization (ExternalAuthn): roles roles = RoleSet { roles: {} }
33892025-04-30T06:43:52.450ZDEBGtest_utilization (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
33902025-04-30T06:43:52.453ZTRACtest_utilization (ExternalAuthn): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = SiloUser { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000000, lookup_type: ById(001de000-5110-4000-8000-000000000000) }, key: 001de000-05e4-4000-8000-000000004007, lookup_type: ById(001de000-05e4-4000-8000-000000004007) }
33912025-04-30T06:43:52.453ZTRACtest_utilization (ExternalAuthn): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. } resource_id = 001de000-5110-4000-8000-000000000000 resource_type = Silo
33922025-04-30T06:43:52.453ZTRACtest_utilization (ExternalAuthn): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database
33932025-04-30T06:43:52.453ZDEBGtest_utilization (ExternalAuthn): roles roles = RoleSet { roles: {} }
33942025-04-30T06:43:52.454ZDEBGtest_utilization (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
33952025-04-30T06:43:52.457ZTRACtest_utilization (ExternalAuthn): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
33962025-04-30T06:43:52.457ZTRACtest_utilization (ExternalAuthn): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database
33972025-04-30T06:43:52.457ZDEBGtest_utilization (ExternalAuthn): roles roles = RoleSet { roles: {} }
33982025-04-30T06:43:52.458ZDEBGtest_utilization (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
33992025-04-30T06:43:52.460ZDEBGtest_utilization (ExternalAuthn): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "external-authenticator")} }
34002025-04-30T06:43:52.464ZDEBGtest_utilization (ExternalAuthn): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = SiloUser { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000000, lookup_type: ById(001de000-5110-4000-8000-000000000000) }, key: 001de000-05e4-4000-8000-000000004007, lookup_type: ById(001de000-05e4-4000-8000-000000004007) } result = Ok(())
34012025-04-30T06:43:52.464ZTRACtest_utilization (ExternalAuthn): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database
34022025-04-30T06:43:52.464ZDEBGtest_utilization (ExternalAuthn): roles roles = RoleSet { roles: {} }
34032025-04-30T06:43:52.464ZDEBGtest_utilization (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
34042025-04-30T06:43:52.468ZTRACtest_utilization (ExternalAuthn): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000000, lookup_type: ById(001de000-5110-4000-8000-000000000000) }
34052025-04-30T06:43:52.468ZTRACtest_utilization (ExternalAuthn): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. } resource_id = 001de000-5110-4000-8000-000000000000 resource_type = Silo
34062025-04-30T06:43:52.468ZTRACtest_utilization (ExternalAuthn): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database
34072025-04-30T06:43:52.468ZDEBGtest_utilization (ExternalAuthn): roles roles = RoleSet { roles: {} }
34082025-04-30T06:43:52.468ZDEBGtest_utilization (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
34092025-04-30T06:43:52.471ZTRACtest_utilization (ExternalAuthn): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
34102025-04-30T06:43:52.471ZTRACtest_utilization (ExternalAuthn): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database
34112025-04-30T06:43:52.471ZDEBGtest_utilization (ExternalAuthn): roles roles = RoleSet { roles: {} }
34122025-04-30T06:43:52.472ZDEBGtest_utilization (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
34132025-04-30T06:43:52.475ZDEBGtest_utilization (ExternalAuthn): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "external-authenticator")} }
34142025-04-30T06:43:52.478ZDEBGtest_utilization (ExternalAuthn): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000000, lookup_type: ById(001de000-5110-4000-8000-000000000000) } result = Ok(())
34152025-04-30T06:43:52.478ZTRACtest_utilization (dropshot_external): authn result: Ok(Context { kind: Authenticated(Details { actor: Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. } }, Some(SiloAuthnPolicy { mapped_fleet_roles: {} })), schemes_tried: [SchemeName("spoof")] }) local_addr = 127.0.0.1:46348 method = GET remote_addr = 127.0.0.1:41283 req_id = c8edc401-8413-432d-a9a3-d491c17d9b13 uri = /v1/system/utilization/silos
34162025-04-30T06:43:52.478ZTRACtest_utilization (dropshot_external): authorize begin action = ListChildren actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true local_addr = 127.0.0.1:46348 method = GET remote_addr = 127.0.0.1:41283 req_id = c8edc401-8413-432d-a9a3-d491c17d9b13 resource = Fleet uri = /v1/system/utilization/silos
34172025-04-30T06:43:52.478ZTRACtest_utilization (dropshot_external): loading roles actor = Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. } actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true local_addr = 127.0.0.1:46348 method = GET remote_addr = 127.0.0.1:41283 req_id = c8edc401-8413-432d-a9a3-d491c17d9b13 resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet uri = /v1/system/utilization/silos
34182025-04-30T06:43:52.478ZTRACtest_utilization (dropshot_external): authorize begin action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true local_addr = 127.0.0.1:46348 method = GET remote_addr = 127.0.0.1:41283 req_id = c8edc401-8413-432d-a9a3-d491c17d9b13 resource = Database uri = /v1/system/utilization/silos
34192025-04-30T06:43:52.478ZDEBGtest_utilization (dropshot_external): roles actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true local_addr = 127.0.0.1:46348 method = GET remote_addr = 127.0.0.1:41283 req_id = c8edc401-8413-432d-a9a3-d491c17d9b13 roles = RoleSet { roles: {} } uri = /v1/system/utilization/silos
34202025-04-30T06:43:52.479ZDEBGtest_utilization (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true local_addr = 127.0.0.1:46348 method = GET remote_addr = 127.0.0.1:41283 req_id = c8edc401-8413-432d-a9a3-d491c17d9b13 resource = Database result = Ok(()) uri = /v1/system/utilization/silos
34212025-04-30T06:43:52.484ZDEBGtest_utilization (dropshot_external): roles actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true local_addr = 127.0.0.1:46348 method = GET remote_addr = 127.0.0.1:41283 req_id = c8edc401-8413-432d-a9a3-d491c17d9b13 roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} } uri = /v1/system/utilization/silos
34222025-04-30T06:43:52.488ZDEBGtest_utilization (dropshot_external): authorize result action = ListChildren actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true local_addr = 127.0.0.1:46348 method = GET remote_addr = 127.0.0.1:41283 req_id = c8edc401-8413-432d-a9a3-d491c17d9b13 resource = Fleet result = Ok(()) uri = /v1/system/utilization/silos
34232025-04-30T06:43:52.488ZTRACtest_utilization (dropshot_external): authorize begin action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true local_addr = 127.0.0.1:46348 method = GET remote_addr = 127.0.0.1:41283 req_id = c8edc401-8413-432d-a9a3-d491c17d9b13 resource = Database uri = /v1/system/utilization/silos
34242025-04-30T06:43:52.489ZDEBGtest_utilization (dropshot_external): roles actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true local_addr = 127.0.0.1:46348 method = GET remote_addr = 127.0.0.1:41283 req_id = c8edc401-8413-432d-a9a3-d491c17d9b13 roles = RoleSet { roles: {} } uri = /v1/system/utilization/silos
34252025-04-30T06:43:52.490ZDEBGtest_utilization (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true local_addr = 127.0.0.1:46348 method = GET remote_addr = 127.0.0.1:41283 req_id = c8edc401-8413-432d-a9a3-d491c17d9b13 resource = Database result = Ok(()) uri = /v1/system/utilization/silos
34262025-04-30T06:43:52.512ZINFOtest_utilization (dropshot_external): request completed latency_us = 68342 local_addr = 127.0.0.1:46348 method = GET remote_addr = 127.0.0.1:41283 req_id = c8edc401-8413-432d-a9a3-d491c17d9b13 response_code = 200 uri = /v1/system/utilization/silos
34272025-04-30T06:43:52.513ZINFOtest_utilization (external client test context): client received response status = 200
34282025-04-30T06:43:52.514ZINFOtest_utilization (external client test context): client request body = Body { inner: BoxBody } method = PUT uri = http://127.0.0.1:46348/v1/system/silos/test-suite-silo/quotas
34292025-04-30T06:43:52.514ZTRACtest_utilization (dropshot_external): incoming request local_addr = 127.0.0.1:46348 method = PUT remote_addr = 127.0.0.1:41283 req_id = 76fbfdb2-fe13-4562-8111-427a723f50cf uri = /v1/system/silos/test-suite-silo/quotas
34302025-04-30T06:43:52.514ZTRACtest_utilization (dropshot_external): authn: trying SchemeName("spoof") local_addr = 127.0.0.1:46348 method = PUT remote_addr = 127.0.0.1:41283 req_id = 76fbfdb2-fe13-4562-8111-427a723f50cf uri = /v1/system/silos/test-suite-silo/quotas
34312025-04-30T06:43:52.515ZDEBGtest_utilization (dropshot_external): looking up silo for user local_addr = 127.0.0.1:46348 method = PUT remote_addr = 127.0.0.1:41283 req_id = 76fbfdb2-fe13-4562-8111-427a723f50cf silo_user_id = 001de000-05e4-4000-8000-000000004007 uri = /v1/system/silos/test-suite-silo/quotas
34322025-04-30T06:43:52.515ZTRACtest_utilization (ExternalAuthn): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database
34332025-04-30T06:43:52.515ZDEBGtest_utilization (ExternalAuthn): roles roles = RoleSet { roles: {} }
34342025-04-30T06:43:52.515ZDEBGtest_utilization (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
34352025-04-30T06:43:52.519ZTRACtest_utilization (ExternalAuthn): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database
34362025-04-30T06:43:52.519ZDEBGtest_utilization (ExternalAuthn): roles roles = RoleSet { roles: {} }
34372025-04-30T06:43:52.520ZDEBGtest_utilization (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
34382025-04-30T06:43:52.522ZTRACtest_utilization (ExternalAuthn): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = SiloUser { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000000, lookup_type: ById(001de000-5110-4000-8000-000000000000) }, key: 001de000-05e4-4000-8000-000000004007, lookup_type: ById(001de000-05e4-4000-8000-000000004007) }
34392025-04-30T06:43:52.522ZTRACtest_utilization (ExternalAuthn): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. } resource_id = 001de000-5110-4000-8000-000000000000 resource_type = Silo
34402025-04-30T06:43:52.522ZTRACtest_utilization (ExternalAuthn): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database
34412025-04-30T06:43:52.522ZDEBGtest_utilization (ExternalAuthn): roles roles = RoleSet { roles: {} }
34422025-04-30T06:43:52.523ZDEBGtest_utilization (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
34432025-04-30T06:43:52.525ZTRACtest_utilization (ExternalAuthn): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
34442025-04-30T06:43:52.525ZTRACtest_utilization (ExternalAuthn): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database
34452025-04-30T06:43:52.525ZDEBGtest_utilization (ExternalAuthn): roles roles = RoleSet { roles: {} }
34462025-04-30T06:43:52.525ZDEBGtest_utilization (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
34472025-04-30T06:43:52.527ZDEBGtest_utilization (ExternalAuthn): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "external-authenticator")} }
34482025-04-30T06:43:52.531ZDEBGtest_utilization (ExternalAuthn): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = SiloUser { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000000, lookup_type: ById(001de000-5110-4000-8000-000000000000) }, key: 001de000-05e4-4000-8000-000000004007, lookup_type: ById(001de000-05e4-4000-8000-000000004007) } result = Ok(())
34492025-04-30T06:43:52.531ZTRACtest_utilization (ExternalAuthn): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database
34502025-04-30T06:43:52.531ZDEBGtest_utilization (ExternalAuthn): roles roles = RoleSet { roles: {} }
34512025-04-30T06:43:52.532ZDEBGtest_utilization (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
34522025-04-30T06:43:52.544ZTRACtest_utilization (ExternalAuthn): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000000, lookup_type: ById(001de000-5110-4000-8000-000000000000) }
34532025-04-30T06:43:52.544ZTRACtest_utilization (ExternalAuthn): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. } resource_id = 001de000-5110-4000-8000-000000000000 resource_type = Silo
34542025-04-30T06:43:52.544ZTRACtest_utilization (ExternalAuthn): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database
34552025-04-30T06:43:52.546ZDEBGtest_utilization (ExternalAuthn): roles roles = RoleSet { roles: {} }
34562025-04-30T06:43:52.547ZDEBGtest_utilization (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
34572025-04-30T06:43:52.554ZTRACtest_utilization (ExternalAuthn): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
34582025-04-30T06:43:52.554ZTRACtest_utilization (ExternalAuthn): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database
34592025-04-30T06:43:52.554ZDEBGtest_utilization (ExternalAuthn): roles roles = RoleSet { roles: {} }
34602025-04-30T06:43:52.554ZDEBGtest_utilization (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
34612025-04-30T06:43:52.556ZDEBGtest_utilization (ExternalAuthn): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "external-authenticator")} }
34622025-04-30T06:43:52.559ZDEBGtest_utilization (ExternalAuthn): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000000, lookup_type: ById(001de000-5110-4000-8000-000000000000) } result = Ok(())
34632025-04-30T06:43:52.559ZTRACtest_utilization (dropshot_external): authn result: Ok(Context { kind: Authenticated(Details { actor: Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. } }, Some(SiloAuthnPolicy { mapped_fleet_roles: {} })), schemes_tried: [SchemeName("spoof")] }) local_addr = 127.0.0.1:46348 method = PUT remote_addr = 127.0.0.1:41283 req_id = 76fbfdb2-fe13-4562-8111-427a723f50cf uri = /v1/system/silos/test-suite-silo/quotas
34642025-04-30T06:43:52.559ZTRACtest_utilization (dropshot_external): authorize begin action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true local_addr = 127.0.0.1:46348 method = PUT remote_addr = 127.0.0.1:41283 req_id = 76fbfdb2-fe13-4562-8111-427a723f50cf resource = Database uri = /v1/system/silos/test-suite-silo/quotas
34652025-04-30T06:43:52.559ZDEBGtest_utilization (dropshot_external): roles actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true local_addr = 127.0.0.1:46348 method = PUT remote_addr = 127.0.0.1:41283 req_id = 76fbfdb2-fe13-4562-8111-427a723f50cf roles = RoleSet { roles: {} } uri = /v1/system/silos/test-suite-silo/quotas
34662025-04-30T06:43:52.560ZDEBGtest_utilization (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true local_addr = 127.0.0.1:46348 method = PUT remote_addr = 127.0.0.1:41283 req_id = 76fbfdb2-fe13-4562-8111-427a723f50cf resource = Database result = Ok(()) uri = /v1/system/silos/test-suite-silo/quotas
34672025-04-30T06:43:52.562ZTRACtest_utilization (dropshot_external): authorize begin action = Modify actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true local_addr = 127.0.0.1:46348 method = PUT remote_addr = 127.0.0.1:41283 req_id = 76fbfdb2-fe13-4562-8111-427a723f50cf resource = Silo { parent: Fleet, key: ad91dcf9-5eb8-4a8d-b721-b5a47ceac136, lookup_type: ByName("test-suite-silo") } uri = /v1/system/silos/test-suite-silo/quotas
34682025-04-30T06:43:52.562ZTRACtest_utilization (dropshot_external): loading roles actor = Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. } actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true local_addr = 127.0.0.1:46348 method = PUT remote_addr = 127.0.0.1:41283 req_id = 76fbfdb2-fe13-4562-8111-427a723f50cf resource_id = ad91dcf9-5eb8-4a8d-b721-b5a47ceac136 resource_type = Silo uri = /v1/system/silos/test-suite-silo/quotas
34692025-04-30T06:43:52.562ZTRACtest_utilization (dropshot_external): authorize begin action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true local_addr = 127.0.0.1:46348 method = PUT remote_addr = 127.0.0.1:41283 req_id = 76fbfdb2-fe13-4562-8111-427a723f50cf resource = Database uri = /v1/system/silos/test-suite-silo/quotas
34702025-04-30T06:43:52.562ZDEBGtest_utilization (dropshot_external): roles actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true local_addr = 127.0.0.1:46348 method = PUT remote_addr = 127.0.0.1:41283 req_id = 76fbfdb2-fe13-4562-8111-427a723f50cf roles = RoleSet { roles: {} } uri = /v1/system/silos/test-suite-silo/quotas
34712025-04-30T06:43:52.563ZDEBGtest_utilization (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true local_addr = 127.0.0.1:46348 method = PUT remote_addr = 127.0.0.1:41283 req_id = 76fbfdb2-fe13-4562-8111-427a723f50cf resource = Database result = Ok(()) uri = /v1/system/silos/test-suite-silo/quotas
34722025-04-30T06:43:52.568ZTRACtest_utilization (dropshot_external): loading roles actor = Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. } actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true local_addr = 127.0.0.1:46348 method = PUT remote_addr = 127.0.0.1:41283 req_id = 76fbfdb2-fe13-4562-8111-427a723f50cf resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet uri = /v1/system/silos/test-suite-silo/quotas
34732025-04-30T06:43:52.568ZTRACtest_utilization (dropshot_external): authorize begin action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true local_addr = 127.0.0.1:46348 method = PUT remote_addr = 127.0.0.1:41283 req_id = 76fbfdb2-fe13-4562-8111-427a723f50cf resource = Database uri = /v1/system/silos/test-suite-silo/quotas
34742025-04-30T06:43:52.568ZDEBGtest_utilization (dropshot_external): roles actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true local_addr = 127.0.0.1:46348 method = PUT remote_addr = 127.0.0.1:41283 req_id = 76fbfdb2-fe13-4562-8111-427a723f50cf roles = RoleSet { roles: {} } uri = /v1/system/silos/test-suite-silo/quotas
34752025-04-30T06:43:52.568ZDEBGtest_utilization (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true local_addr = 127.0.0.1:46348 method = PUT remote_addr = 127.0.0.1:41283 req_id = 76fbfdb2-fe13-4562-8111-427a723f50cf resource = Database result = Ok(()) uri = /v1/system/silos/test-suite-silo/quotas
34762025-04-30T06:43:52.572ZDEBGtest_utilization (dropshot_external): roles actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true local_addr = 127.0.0.1:46348 method = PUT remote_addr = 127.0.0.1:41283 req_id = 76fbfdb2-fe13-4562-8111-427a723f50cf roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} } uri = /v1/system/silos/test-suite-silo/quotas
34772025-04-30T06:43:52.577ZDEBGtest_utilization (dropshot_external): authorize result action = Modify actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true local_addr = 127.0.0.1:46348 method = PUT remote_addr = 127.0.0.1:41283 req_id = 76fbfdb2-fe13-4562-8111-427a723f50cf resource = Silo { parent: Fleet, key: ad91dcf9-5eb8-4a8d-b721-b5a47ceac136, lookup_type: ByName("test-suite-silo") } result = Ok(()) uri = /v1/system/silos/test-suite-silo/quotas
34782025-04-30T06:43:52.577ZTRACtest_utilization (dropshot_external): authorize begin action = Modify actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true local_addr = 127.0.0.1:46348 method = PUT remote_addr = 127.0.0.1:41283 req_id = 76fbfdb2-fe13-4562-8111-427a723f50cf resource = Silo { parent: Fleet, key: ad91dcf9-5eb8-4a8d-b721-b5a47ceac136, lookup_type: ByName("test-suite-silo") } uri = /v1/system/silos/test-suite-silo/quotas
34792025-04-30T06:43:52.577ZTRACtest_utilization (dropshot_external): loading roles actor = Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. } actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true local_addr = 127.0.0.1:46348 method = PUT remote_addr = 127.0.0.1:41283 req_id = 76fbfdb2-fe13-4562-8111-427a723f50cf resource_id = ad91dcf9-5eb8-4a8d-b721-b5a47ceac136 resource_type = Silo uri = /v1/system/silos/test-suite-silo/quotas
34802025-04-30T06:43:52.577ZTRACtest_utilization (dropshot_external): authorize begin action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true local_addr = 127.0.0.1:46348 method = PUT remote_addr = 127.0.0.1:41283 req_id = 76fbfdb2-fe13-4562-8111-427a723f50cf resource = Database uri = /v1/system/silos/test-suite-silo/quotas
34812025-04-30T06:43:52.577ZDEBGtest_utilization (dropshot_external): roles actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true local_addr = 127.0.0.1:46348 method = PUT remote_addr = 127.0.0.1:41283 req_id = 76fbfdb2-fe13-4562-8111-427a723f50cf roles = RoleSet { roles: {} } uri = /v1/system/silos/test-suite-silo/quotas
34822025-04-30T06:43:52.577ZDEBGtest_utilization (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true local_addr = 127.0.0.1:46348 method = PUT remote_addr = 127.0.0.1:41283 req_id = 76fbfdb2-fe13-4562-8111-427a723f50cf resource = Database result = Ok(()) uri = /v1/system/silos/test-suite-silo/quotas
34832025-04-30T06:43:52.580ZTRACtest_utilization (dropshot_external): loading roles actor = Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. } actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true local_addr = 127.0.0.1:46348 method = PUT remote_addr = 127.0.0.1:41283 req_id = 76fbfdb2-fe13-4562-8111-427a723f50cf resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet uri = /v1/system/silos/test-suite-silo/quotas
34842025-04-30T06:43:52.581ZTRACtest_utilization (dropshot_external): authorize begin action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true local_addr = 127.0.0.1:46348 method = PUT remote_addr = 127.0.0.1:41283 req_id = 76fbfdb2-fe13-4562-8111-427a723f50cf resource = Database uri = /v1/system/silos/test-suite-silo/quotas
34852025-04-30T06:43:52.581ZDEBGtest_utilization (dropshot_external): roles actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true local_addr = 127.0.0.1:46348 method = PUT remote_addr = 127.0.0.1:41283 req_id = 76fbfdb2-fe13-4562-8111-427a723f50cf roles = RoleSet { roles: {} } uri = /v1/system/silos/test-suite-silo/quotas
34862025-04-30T06:43:52.581ZDEBGtest_utilization (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true local_addr = 127.0.0.1:46348 method = PUT remote_addr = 127.0.0.1:41283 req_id = 76fbfdb2-fe13-4562-8111-427a723f50cf resource = Database result = Ok(()) uri = /v1/system/silos/test-suite-silo/quotas
34872025-04-30T06:43:52.586ZDEBGtest_utilization (dropshot_external): roles actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true local_addr = 127.0.0.1:46348 method = PUT remote_addr = 127.0.0.1:41283 req_id = 76fbfdb2-fe13-4562-8111-427a723f50cf roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} } uri = /v1/system/silos/test-suite-silo/quotas
34882025-04-30T06:43:52.600ZDEBGtest_utilization (dropshot_external): authorize result action = Modify actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true local_addr = 127.0.0.1:46348 method = PUT remote_addr = 127.0.0.1:41283 req_id = 76fbfdb2-fe13-4562-8111-427a723f50cf resource = Silo { parent: Fleet, key: ad91dcf9-5eb8-4a8d-b721-b5a47ceac136, lookup_type: ByName("test-suite-silo") } result = Ok(()) uri = /v1/system/silos/test-suite-silo/quotas
34892025-04-30T06:43:52.600ZTRACtest_utilization (dropshot_external): authorize begin action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true local_addr = 127.0.0.1:46348 method = PUT remote_addr = 127.0.0.1:41283 req_id = 76fbfdb2-fe13-4562-8111-427a723f50cf resource = Database uri = /v1/system/silos/test-suite-silo/quotas
34902025-04-30T06:43:52.600ZDEBGtest_utilization (dropshot_external): roles actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true local_addr = 127.0.0.1:46348 method = PUT remote_addr = 127.0.0.1:41283 req_id = 76fbfdb2-fe13-4562-8111-427a723f50cf roles = RoleSet { roles: {} } uri = /v1/system/silos/test-suite-silo/quotas
34912025-04-30T06:43:52.600ZDEBGtest_utilization (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true local_addr = 127.0.0.1:46348 method = PUT remote_addr = 127.0.0.1:41283 req_id = 76fbfdb2-fe13-4562-8111-427a723f50cf resource = Database result = Ok(()) uri = /v1/system/silos/test-suite-silo/quotas
34922025-04-30T06:43:52.605ZINFOtest_utilization (dropshot_external): request completed latency_us = 90321 local_addr = 127.0.0.1:46348 method = PUT remote_addr = 127.0.0.1:41283 req_id = 76fbfdb2-fe13-4562-8111-427a723f50cf response_code = 200 uri = /v1/system/silos/test-suite-silo/quotas
34932025-04-30T06:43:52.605ZINFOtest_utilization (external client test context): client received response status = 200
34942025-04-30T06:43:52.606ZINFOtest_utilization (external client test context): client request body = Body { inner: BoxBody } method = GET uri = http://127.0.0.1:46348/v1/system/utilization/silos
34952025-04-30T06:43:52.612ZTRACtest_utilization (dropshot_external): incoming request local_addr = 127.0.0.1:46348 method = GET remote_addr = 127.0.0.1:41283 req_id = 2d90fce5-cfcc-4ae7-a1a3-333c63f19b32 uri = /v1/system/utilization/silos
34962025-04-30T06:43:52.612ZTRACtest_utilization (dropshot_external): authn: trying SchemeName("spoof") local_addr = 127.0.0.1:46348 method = GET remote_addr = 127.0.0.1:41283 req_id = 2d90fce5-cfcc-4ae7-a1a3-333c63f19b32 uri = /v1/system/utilization/silos
34972025-04-30T06:43:52.612ZDEBGtest_utilization (dropshot_external): looking up silo for user local_addr = 127.0.0.1:46348 method = GET remote_addr = 127.0.0.1:41283 req_id = 2d90fce5-cfcc-4ae7-a1a3-333c63f19b32 silo_user_id = 001de000-05e4-4000-8000-000000004007 uri = /v1/system/utilization/silos
34982025-04-30T06:43:52.612ZTRACtest_utilization (ExternalAuthn): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database
34992025-04-30T06:43:52.613ZDEBGtest_utilization (ExternalAuthn): roles roles = RoleSet { roles: {} }
35002025-04-30T06:43:52.613ZDEBGtest_utilization (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
35012025-04-30T06:43:52.617ZTRACtest_utilization (ExternalAuthn): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database
35022025-04-30T06:43:52.617ZDEBGtest_utilization (ExternalAuthn): roles roles = RoleSet { roles: {} }
35032025-04-30T06:43:52.618ZDEBGtest_utilization (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
35042025-04-30T06:43:52.624ZTRACtest_utilization (ExternalAuthn): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = SiloUser { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000000, lookup_type: ById(001de000-5110-4000-8000-000000000000) }, key: 001de000-05e4-4000-8000-000000004007, lookup_type: ById(001de000-05e4-4000-8000-000000004007) }
35052025-04-30T06:43:52.624ZTRACtest_utilization (ExternalAuthn): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. } resource_id = 001de000-5110-4000-8000-000000000000 resource_type = Silo
35062025-04-30T06:43:52.624ZTRACtest_utilization (ExternalAuthn): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database
35072025-04-30T06:43:52.624ZDEBGtest_utilization (ExternalAuthn): roles roles = RoleSet { roles: {} }
35082025-04-30T06:43:52.625ZDEBGtest_utilization (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
35092025-04-30T06:43:52.628ZTRACtest_utilization (ExternalAuthn): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
35102025-04-30T06:43:52.629ZTRACtest_utilization (ExternalAuthn): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database
35112025-04-30T06:43:52.629ZDEBGtest_utilization (ExternalAuthn): roles roles = RoleSet { roles: {} }
35122025-04-30T06:43:52.629ZDEBGtest_utilization (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
35132025-04-30T06:43:52.640ZDEBGtest_utilization (ExternalAuthn): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "external-authenticator")} }
35142025-04-30T06:43:52.644ZDEBGtest_utilization (ExternalAuthn): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = SiloUser { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000000, lookup_type: ById(001de000-5110-4000-8000-000000000000) }, key: 001de000-05e4-4000-8000-000000004007, lookup_type: ById(001de000-05e4-4000-8000-000000004007) } result = Ok(())
35152025-04-30T06:43:52.644ZTRACtest_utilization (ExternalAuthn): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database
35162025-04-30T06:43:52.644ZDEBGtest_utilization (ExternalAuthn): roles roles = RoleSet { roles: {} }
35172025-04-30T06:43:52.645ZDEBGtest_utilization (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
35182025-04-30T06:43:52.660ZTRACtest_utilization (ExternalAuthn): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000000, lookup_type: ById(001de000-5110-4000-8000-000000000000) }
35192025-04-30T06:43:52.660ZTRACtest_utilization (ExternalAuthn): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. } resource_id = 001de000-5110-4000-8000-000000000000 resource_type = Silo
35202025-04-30T06:43:52.660ZTRACtest_utilization (ExternalAuthn): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database
35212025-04-30T06:43:52.660ZDEBGtest_utilization (ExternalAuthn): roles roles = RoleSet { roles: {} }
35222025-04-30T06:43:52.661ZDEBGtest_utilization (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
35232025-04-30T06:43:52.667ZTRACtest_utilization (ExternalAuthn): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
35242025-04-30T06:43:52.667ZTRACtest_utilization (ExternalAuthn): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database
35252025-04-30T06:43:52.667ZDEBGtest_utilization (ExternalAuthn): roles roles = RoleSet { roles: {} }
35262025-04-30T06:43:52.675ZDEBGtest_utilization (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
35272025-04-30T06:43:52.678ZDEBGtest_utilization (ExternalAuthn): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "external-authenticator")} }
35282025-04-30T06:43:52.681ZDEBGtest_utilization (ExternalAuthn): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000000, lookup_type: ById(001de000-5110-4000-8000-000000000000) } result = Ok(())
35292025-04-30T06:43:52.681ZTRACtest_utilization (dropshot_external): authn result: Ok(Context { kind: Authenticated(Details { actor: Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. } }, Some(SiloAuthnPolicy { mapped_fleet_roles: {} })), schemes_tried: [SchemeName("spoof")] }) local_addr = 127.0.0.1:46348 method = GET remote_addr = 127.0.0.1:41283 req_id = 2d90fce5-cfcc-4ae7-a1a3-333c63f19b32 uri = /v1/system/utilization/silos
35302025-04-30T06:43:52.681ZTRACtest_utilization (dropshot_external): authorize begin action = ListChildren actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true local_addr = 127.0.0.1:46348 method = GET remote_addr = 127.0.0.1:41283 req_id = 2d90fce5-cfcc-4ae7-a1a3-333c63f19b32 resource = Fleet uri = /v1/system/utilization/silos
35312025-04-30T06:43:52.681ZTRACtest_utilization (dropshot_external): loading roles actor = Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. } actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true local_addr = 127.0.0.1:46348 method = GET remote_addr = 127.0.0.1:41283 req_id = 2d90fce5-cfcc-4ae7-a1a3-333c63f19b32 resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet uri = /v1/system/utilization/silos
35322025-04-30T06:43:52.681ZTRACtest_utilization (dropshot_external): authorize begin action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true local_addr = 127.0.0.1:46348 method = GET remote_addr = 127.0.0.1:41283 req_id = 2d90fce5-cfcc-4ae7-a1a3-333c63f19b32 resource = Database uri = /v1/system/utilization/silos
35332025-04-30T06:43:52.681ZDEBGtest_utilization (dropshot_external): roles actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true local_addr = 127.0.0.1:46348 method = GET remote_addr = 127.0.0.1:41283 req_id = 2d90fce5-cfcc-4ae7-a1a3-333c63f19b32 roles = RoleSet { roles: {} } uri = /v1/system/utilization/silos
35342025-04-30T06:43:52.682ZDEBGtest_utilization (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true local_addr = 127.0.0.1:46348 method = GET remote_addr = 127.0.0.1:41283 req_id = 2d90fce5-cfcc-4ae7-a1a3-333c63f19b32 resource = Database result = Ok(()) uri = /v1/system/utilization/silos
35352025-04-30T06:43:52.692ZDEBGtest_utilization (dropshot_external): roles actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true local_addr = 127.0.0.1:46348 method = GET remote_addr = 127.0.0.1:41283 req_id = 2d90fce5-cfcc-4ae7-a1a3-333c63f19b32 roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} } uri = /v1/system/utilization/silos
35362025-04-30T06:43:52.696ZDEBGtest_utilization (dropshot_external): authorize result action = ListChildren actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true local_addr = 127.0.0.1:46348 method = GET remote_addr = 127.0.0.1:41283 req_id = 2d90fce5-cfcc-4ae7-a1a3-333c63f19b32 resource = Fleet result = Ok(()) uri = /v1/system/utilization/silos
35372025-04-30T06:43:52.696ZTRACtest_utilization (dropshot_external): authorize begin action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true local_addr = 127.0.0.1:46348 method = GET remote_addr = 127.0.0.1:41283 req_id = 2d90fce5-cfcc-4ae7-a1a3-333c63f19b32 resource = Database uri = /v1/system/utilization/silos
35382025-04-30T06:43:52.696ZDEBGtest_utilization (dropshot_external): roles actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true local_addr = 127.0.0.1:46348 method = GET remote_addr = 127.0.0.1:41283 req_id = 2d90fce5-cfcc-4ae7-a1a3-333c63f19b32 roles = RoleSet { roles: {} } uri = /v1/system/utilization/silos
35392025-04-30T06:43:52.697ZDEBGtest_utilization (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true local_addr = 127.0.0.1:46348 method = GET remote_addr = 127.0.0.1:41283 req_id = 2d90fce5-cfcc-4ae7-a1a3-333c63f19b32 resource = Database result = Ok(()) uri = /v1/system/utilization/silos
35402025-04-30T06:43:52.703ZINFOtest_utilization (dropshot_external): request completed latency_us = 91011 local_addr = 127.0.0.1:46348 method = GET remote_addr = 127.0.0.1:41283 req_id = 2d90fce5-cfcc-4ae7-a1a3-333c63f19b32 response_code = 200 uri = /v1/system/utilization/silos
35412025-04-30T06:43:52.704ZINFOtest_utilization (external client test context): client received response status = 200
35422025-04-30T06:43:52.704ZINFOtest_utilization (external client test context): client request body = Body { inner: BoxBody } method = POST uri = http://127.0.0.1:46348/v1/projects
35432025-04-30T06:43:52.705ZTRACtest_utilization (dropshot_external): incoming request local_addr = 127.0.0.1:46348 method = POST remote_addr = 127.0.0.1:41283 req_id = 07ae243d-4bd1-44d6-8cea-cee37952d5a7 uri = /v1/projects
35442025-04-30T06:43:52.705ZTRACtest_utilization (dropshot_external): authn: trying SchemeName("spoof") local_addr = 127.0.0.1:46348 method = POST remote_addr = 127.0.0.1:41283 req_id = 07ae243d-4bd1-44d6-8cea-cee37952d5a7 uri = /v1/projects
35452025-04-30T06:43:52.705ZDEBGtest_utilization (dropshot_external): looking up silo for user local_addr = 127.0.0.1:46348 method = POST remote_addr = 127.0.0.1:41283 req_id = 07ae243d-4bd1-44d6-8cea-cee37952d5a7 silo_user_id = 001de000-05e4-4000-8000-000000004007 uri = /v1/projects
35462025-04-30T06:43:52.705ZTRACtest_utilization (ExternalAuthn): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database
35472025-04-30T06:43:52.705ZDEBGtest_utilization (ExternalAuthn): roles roles = RoleSet { roles: {} }
35482025-04-30T06:43:52.706ZDEBGtest_utilization (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
35492025-04-30T06:43:52.710ZTRACtest_utilization (ExternalAuthn): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database
35502025-04-30T06:43:52.710ZDEBGtest_utilization (ExternalAuthn): roles roles = RoleSet { roles: {} }
35512025-04-30T06:43:52.712ZDEBGtest_utilization (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
35522025-04-30T06:43:52.723ZTRACtest_utilization (ExternalAuthn): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = SiloUser { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000000, lookup_type: ById(001de000-5110-4000-8000-000000000000) }, key: 001de000-05e4-4000-8000-000000004007, lookup_type: ById(001de000-05e4-4000-8000-000000004007) }
35532025-04-30T06:43:52.723ZTRACtest_utilization (ExternalAuthn): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. } resource_id = 001de000-5110-4000-8000-000000000000 resource_type = Silo
35542025-04-30T06:43:52.723ZTRACtest_utilization (ExternalAuthn): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database
35552025-04-30T06:43:52.723ZDEBGtest_utilization (ExternalAuthn): roles roles = RoleSet { roles: {} }
35562025-04-30T06:43:52.724ZDEBGtest_utilization (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
35572025-04-30T06:43:52.732ZTRACtest_utilization (ExternalAuthn): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
35582025-04-30T06:43:52.740ZTRACtest_utilization (ExternalAuthn): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database
35592025-04-30T06:43:52.740ZDEBGtest_utilization (ExternalAuthn): roles roles = RoleSet { roles: {} }
35602025-04-30T06:43:52.757ZDEBGtest_utilization (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
35612025-04-30T06:43:52.761ZDEBGtest_utilization (ExternalAuthn): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "external-authenticator")} }
35622025-04-30T06:43:52.769ZDEBGtest_utilization (ExternalAuthn): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = SiloUser { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000000, lookup_type: ById(001de000-5110-4000-8000-000000000000) }, key: 001de000-05e4-4000-8000-000000004007, lookup_type: ById(001de000-05e4-4000-8000-000000004007) } result = Ok(())
35632025-04-30T06:43:52.769ZTRACtest_utilization (ExternalAuthn): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database
35642025-04-30T06:43:52.769ZDEBGtest_utilization (ExternalAuthn): roles roles = RoleSet { roles: {} }
35652025-04-30T06:43:52.770ZDEBGtest_utilization (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
35662025-04-30T06:43:52.776ZTRACtest_utilization (ExternalAuthn): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000000, lookup_type: ById(001de000-5110-4000-8000-000000000000) }
35672025-04-30T06:43:52.776ZTRACtest_utilization (ExternalAuthn): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. } resource_id = 001de000-5110-4000-8000-000000000000 resource_type = Silo
35682025-04-30T06:43:52.776ZTRACtest_utilization (ExternalAuthn): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database
35692025-04-30T06:43:52.776ZDEBGtest_utilization (ExternalAuthn): roles roles = RoleSet { roles: {} }
35702025-04-30T06:43:52.777ZDEBGtest_utilization (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
35712025-04-30T06:43:52.780ZTRACtest_utilization (ExternalAuthn): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
35722025-04-30T06:43:52.780ZTRACtest_utilization (ExternalAuthn): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database
35732025-04-30T06:43:52.780ZDEBGtest_utilization (ExternalAuthn): roles roles = RoleSet { roles: {} }
35742025-04-30T06:43:52.782ZDEBGtest_utilization (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
35752025-04-30T06:43:52.786ZDEBGtest_utilization (ExternalAuthn): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "external-authenticator")} }
35762025-04-30T06:43:52.791ZDEBGtest_utilization (ExternalAuthn): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000000, lookup_type: ById(001de000-5110-4000-8000-000000000000) } result = Ok(())
35772025-04-30T06:43:52.791ZTRACtest_utilization (dropshot_external): authn result: Ok(Context { kind: Authenticated(Details { actor: Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. } }, Some(SiloAuthnPolicy { mapped_fleet_roles: {} })), schemes_tried: [SchemeName("spoof")] }) local_addr = 127.0.0.1:46348 method = POST remote_addr = 127.0.0.1:41283 req_id = 07ae243d-4bd1-44d6-8cea-cee37952d5a7 uri = /v1/projects
35782025-04-30T06:43:52.791ZTRACtest_utilization (dropshot_external): authorize begin action = CreateChild actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true local_addr = 127.0.0.1:46348 method = POST remote_addr = 127.0.0.1:41283 req_id = 07ae243d-4bd1-44d6-8cea-cee37952d5a7 resource = Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000000, lookup_type: ById(001de000-5110-4000-8000-000000000000) } uri = /v1/projects
35792025-04-30T06:43:52.791ZTRACtest_utilization (dropshot_external): loading roles actor = Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. } actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true local_addr = 127.0.0.1:46348 method = POST remote_addr = 127.0.0.1:41283 req_id = 07ae243d-4bd1-44d6-8cea-cee37952d5a7 resource_id = 001de000-5110-4000-8000-000000000000 resource_type = Silo uri = /v1/projects
35802025-04-30T06:43:52.791ZTRACtest_utilization (dropshot_external): authorize begin action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true local_addr = 127.0.0.1:46348 method = POST remote_addr = 127.0.0.1:41283 req_id = 07ae243d-4bd1-44d6-8cea-cee37952d5a7 resource = Database uri = /v1/projects
35812025-04-30T06:43:52.791ZDEBGtest_utilization (dropshot_external): roles actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true local_addr = 127.0.0.1:46348 method = POST remote_addr = 127.0.0.1:41283 req_id = 07ae243d-4bd1-44d6-8cea-cee37952d5a7 roles = RoleSet { roles: {} } uri = /v1/projects
35822025-04-30T06:43:52.792ZDEBGtest_utilization (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true local_addr = 127.0.0.1:46348 method = POST remote_addr = 127.0.0.1:41283 req_id = 07ae243d-4bd1-44d6-8cea-cee37952d5a7 resource = Database result = Ok(()) uri = /v1/projects
35832025-04-30T06:43:52.796ZTRACtest_utilization (dropshot_external): loading roles actor = Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. } actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true local_addr = 127.0.0.1:46348 method = POST remote_addr = 127.0.0.1:41283 req_id = 07ae243d-4bd1-44d6-8cea-cee37952d5a7 resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet uri = /v1/projects
35842025-04-30T06:43:52.796ZTRACtest_utilization (dropshot_external): authorize begin action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true local_addr = 127.0.0.1:46348 method = POST remote_addr = 127.0.0.1:41283 req_id = 07ae243d-4bd1-44d6-8cea-cee37952d5a7 resource = Database uri = /v1/projects
35852025-04-30T06:43:52.796ZDEBGtest_utilization (dropshot_external): roles actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true local_addr = 127.0.0.1:46348 method = POST remote_addr = 127.0.0.1:41283 req_id = 07ae243d-4bd1-44d6-8cea-cee37952d5a7 roles = RoleSet { roles: {} } uri = /v1/projects
35862025-04-30T06:43:52.798ZDEBGtest_utilization (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true local_addr = 127.0.0.1:46348 method = POST remote_addr = 127.0.0.1:41283 req_id = 07ae243d-4bd1-44d6-8cea-cee37952d5a7 resource = Database result = Ok(()) uri = /v1/projects
35872025-04-30T06:43:52.802ZDEBGtest_utilization (dropshot_external): roles actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true local_addr = 127.0.0.1:46348 method = POST remote_addr = 127.0.0.1:41283 req_id = 07ae243d-4bd1-44d6-8cea-cee37952d5a7 roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin"), (Silo, 001de000-5110-4000-8000-000000000000, "admin")} } uri = /v1/projects
35882025-04-30T06:43:52.809ZDEBGtest_utilization (dropshot_external): authorize result action = CreateChild actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true local_addr = 127.0.0.1:46348 method = POST remote_addr = 127.0.0.1:41283 req_id = 07ae243d-4bd1-44d6-8cea-cee37952d5a7 resource = Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000000, lookup_type: ById(001de000-5110-4000-8000-000000000000) } result = Ok(()) uri = /v1/projects
35892025-04-30T06:43:52.810ZINFOtest_utilization (SagaExecutor): preparing saga saga_id = dd4f75ce-0539-4224-8fc6-8244063d4551 saga_name = project-create
35902025-04-30T06:43:52.810ZINFOtest_utilization (SEC): saga create dag = {"end_node":21,"graph":{"edge_property":"directed","edges":[[0,1,null],[1,2,null],[3,4,null],[4,5,null],[5,6,null],[6,7,null],[7,8,null],[8,9,null],[9,10,null],[10,11,null],[11,12,null],[12,13,null],[13,14,null],[14,15,null],[15,16,null],[16,17,null],[17,18,null],[2,3,null],[18,19,null],[20,0,null],[19,21,null]],"node_holes":[],"nodes":[{"Action":{"action_name":"project_create.project_create_record","label":"ProjectCreateRecord","name":"project"}},{"Action":{"action_name":"project_create.project_create_vpc_params","label":"ProjectCreateVpcParams","name":"vpc_create_params"}},{"SubsagaStart":{"params_node_name":"vpc_create_params","saga_name":"vpc-create"}},{"Action":{"action_name":"common.uuid_generate","label":"GenerateVpcId","name":"vpc_id"}},{"Action":{"action_name":"common.uuid_generate","label":"GenerateSystemRouterId","name":"system_router_id"}},{"Action":{"action_name":"common.uuid_generate","label":"GenerateDefaultV4RouteId","name":"default_v4_route_id"}},{"Action":{"action_name":"common.uuid_generate","label":"GenerateDefaultV6RouteId","name":"default_v6_route_id"}},{"Action":{"action_name":"common.uuid_generate","label":"GenerateSubnetRouteId","name":"subnet_route_id"}},{"Action":{"action_name":"common.uuid_generate","label":"GenerateDefaultSubnetId","name":"default_subnet_id"}},{"Action":{"action_name":"common.uuid_generate","label":"GenerateDefaultInternetGatewayId","name":"default_internet_gateway_id"}},{"Action":{"action_name":"vpc_create.vpc_create_vpc","label":"VpcCreateVpc","name":"vpc"}},{"Action":{"action_name":"vpc_create.vpc_create_router","label":"VpcCreateRouter","name":"router"}},{"Action":{"action_name":"vpc_create.vpc_create_v4_route","label":"VpcCreateV4Route","name":"route4"}},{"Action":{"action_name":"vpc_create.vpc_create_v6_route","label":"VpcCreateV6Route","name":"route6"}},{"Action":{"action_name":"vpc_create.vpc_create_subnet","label":"VpcCreateSubnet","name":"subnet"}},{"Action":{"action_name":"vpc_create.vpc_create_subnet_route","label":"VpcCreateSubnetRoute","name":"route"}},{"Action":{"action_name":"vpc_create.vpc_update_firewall","label":"VpcUpdateFirewall","name":"firewall"}},{"Action":{"action_name":"vpc_create.vpc_create_gateway","label":"VpcCreateGateway","name":"gateway"}},{"Action":{"action_name":"vpc_create.vpc_notify_sleds","label":"VpcNotifySleds","name":"no_result"}},{"SubsagaEnd":{"name":"vpc"}},{"Start":{"params":{"authz_silo":{"key":"001de000-5110-4000-8000-000000000000","lookup_type":{"ById":"001de000-5110-4000-8000-000000000000"},"parent":null},"project_create":{"description":"a pier","name":"utilization-test-project"},"serialized_authn":{"kind":{"Authenticated":[{"actor":{"SiloUser":{"silo_id":"001de000-5110-4000-8000-000000000000","silo_user_id":"001de000-05e4-4000-8000-000000004007"}}},{"mapped_fleet_roles":{}}]}}}}},"End"]},"saga_name":"project-create","start_node":20} saga_id = dd4f75ce-0539-4224-8fc6-8244063d4551 saga_name = project-create sec_id = ac55d8d9-a298-4534-898b-beee1c9e4e52
35912025-04-30T06:43:52.810ZINFOtest_utilization (SecStore): creating saga saga_id = dd4f75ce-0539-4224-8fc6-8244063d4551 saga_name = project-create
35922025-04-30T06:43:52.816ZDEBGtest_utilization (producer-registration-task): will register with Nexus at 127.0.0.1:32956
35932025-04-30T06:43:52.873ZDEBGtest_utilization (producer-registration-task): client request body = Some(Body) method = POST uri = http://127.0.0.1:32956/metrics/producers
35942025-04-30T06:43:52.874ZINFOtest_utilization (SagaExecutor): starting saga saga_id = dd4f75ce-0539-4224-8fc6-8244063d4551 saga_name = project-create
35952025-04-30T06:43:52.874ZINFOtest_utilization (SEC): saga start saga_id = dd4f75ce-0539-4224-8fc6-8244063d4551 saga_name = project-create sec_id = ac55d8d9-a298-4534-898b-beee1c9e4e52
35962025-04-30T06:43:52.874ZDEBGtest_utilization (SEC): saga log event new_state = N020 started sec_id = ac55d8d9-a298-4534-898b-beee1c9e4e52
35972025-04-30T06:43:52.874ZDEBGtest_utilization (SecStore): recording saga event event_type = Started node_id = 20 saga_id = dd4f75ce-0539-4224-8fc6-8244063d4551
35982025-04-30T06:43:52.875ZINFOtest_utilization (dropshot_internal): accepted connection local_addr = 127.0.0.1:32956 remote_addr = 127.0.0.1:58244
35992025-04-30T06:43:52.875ZTRACtest_utilization (dropshot_internal): incoming request local_addr = 127.0.0.1:32956 method = POST remote_addr = 127.0.0.1:58244 req_id = a4204a2b-af27-4359-933e-d08a2a9cebe2 uri = /metrics/producers
36002025-04-30T06:43:52.876ZTRACtest_utilization (dropshot_internal): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = 127.0.0.1:32956 method = POST remote_addr = 127.0.0.1:58244 req_id = a4204a2b-af27-4359-933e-d08a2a9cebe2 resource = Database uri = /metrics/producers
36012025-04-30T06:43:52.876ZDEBGtest_utilization (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = 127.0.0.1:32956 method = POST remote_addr = 127.0.0.1:58244 req_id = a4204a2b-af27-4359-933e-d08a2a9cebe2 roles = RoleSet { roles: {} } uri = /metrics/producers
36022025-04-30T06:43:52.877ZDEBGtest_utilization (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = 127.0.0.1:32956 method = POST remote_addr = 127.0.0.1:58244 req_id = a4204a2b-af27-4359-933e-d08a2a9cebe2 resource = Database result = Ok(()) uri = /metrics/producers
36032025-04-30T06:43:52.884ZINFOtest_utilization (nexus): assigned collector to new producer collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 producer_id = ac55d8d9-a298-4534-898b-beee1c9e4e52
36042025-04-30T06:43:52.885ZINFOtest_utilization (dropshot_internal): request completed latency_us = 9561 local_addr = 127.0.0.1:32956 method = POST remote_addr = 127.0.0.1:58244 req_id = a4204a2b-af27-4359-933e-d08a2a9cebe2 response_code = 201 uri = /metrics/producers
36052025-04-30T06:43:52.885ZDEBGtest_utilization (producer-registration-task): client response result = Ok(Response { url: "http://127.0.0.1:32956/metrics/producers", status: 201, headers: {"content-type": "application/json", "x-request-id": "a4204a2b-af27-4359-933e-d08a2a9cebe2", "content-length": "41", "date": "Wed, 30 Apr 2025 06:43:51 GMT"} })
36062025-04-30T06:43:52.887ZDEBGtest_utilization (producer-registration-task): successfully registered with Nexus, pausing until time to renew lease lease_duration = 600s wait_period = 150s
36072025-04-30T06:43:52.888ZDEBGtest_utilization (SEC): saga log event new_state = N020 succeeded sec_id = ac55d8d9-a298-4534-898b-beee1c9e4e52
36082025-04-30T06:43:52.888ZDEBGtest_utilization (SecStore): recording saga event event_type = Succeeded(Null) node_id = 20 saga_id = dd4f75ce-0539-4224-8fc6-8244063d4551
36092025-04-30T06:43:52.892ZDEBGtest_utilization (SEC): saga log event new_state = N000 started sec_id = ac55d8d9-a298-4534-898b-beee1c9e4e52
36102025-04-30T06:43:52.892ZDEBGtest_utilization (SecStore): recording saga event event_type = Started node_id = 0 saga_id = dd4f75ce-0539-4224-8fc6-8244063d4551
36112025-04-30T06:43:52.898ZTRACtest_utilization (SagaExecutor): authorize begin action = CreateChild actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true resource = Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000000, lookup_type: ById(001de000-5110-4000-8000-000000000000) } saga_id = dd4f75ce-0539-4224-8fc6-8244063d4551 saga_name = project-create saga_node = ProjectCreateRecord
36122025-04-30T06:43:52.899ZTRACtest_utilization (SagaExecutor): loading roles actor = Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. } actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true resource_id = 001de000-5110-4000-8000-000000000000 resource_type = Silo saga_id = dd4f75ce-0539-4224-8fc6-8244063d4551 saga_name = project-create saga_node = ProjectCreateRecord
36132025-04-30T06:43:52.899ZTRACtest_utilization (SagaExecutor): authorize begin action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true resource = Database saga_id = dd4f75ce-0539-4224-8fc6-8244063d4551 saga_name = project-create saga_node = ProjectCreateRecord
36142025-04-30T06:43:52.899ZDEBGtest_utilization (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true roles = RoleSet { roles: {} } saga_id = dd4f75ce-0539-4224-8fc6-8244063d4551 saga_name = project-create saga_node = ProjectCreateRecord
36152025-04-30T06:43:52.906ZDEBGtest_utilization (SagaExecutor): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true resource = Database result = Ok(()) saga_id = dd4f75ce-0539-4224-8fc6-8244063d4551 saga_name = project-create saga_node = ProjectCreateRecord
36162025-04-30T06:43:52.909ZTRACtest_utilization (SagaExecutor): loading roles actor = Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. } actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet saga_id = dd4f75ce-0539-4224-8fc6-8244063d4551 saga_name = project-create saga_node = ProjectCreateRecord
36172025-04-30T06:43:52.909ZTRACtest_utilization (SagaExecutor): authorize begin action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true resource = Database saga_id = dd4f75ce-0539-4224-8fc6-8244063d4551 saga_name = project-create saga_node = ProjectCreateRecord
36182025-04-30T06:43:52.909ZDEBGtest_utilization (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true roles = RoleSet { roles: {} } saga_id = dd4f75ce-0539-4224-8fc6-8244063d4551 saga_name = project-create saga_node = ProjectCreateRecord
36192025-04-30T06:43:52.910ZDEBGtest_utilization (SagaExecutor): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true resource = Database result = Ok(()) saga_id = dd4f75ce-0539-4224-8fc6-8244063d4551 saga_name = project-create saga_node = ProjectCreateRecord
36202025-04-30T06:43:52.913ZDEBGtest_utilization (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin"), (Silo, 001de000-5110-4000-8000-000000000000, "admin")} } saga_id = dd4f75ce-0539-4224-8fc6-8244063d4551 saga_name = project-create saga_node = ProjectCreateRecord
36212025-04-30T06:43:52.921ZDEBGtest_utilization (SagaExecutor): authorize result action = CreateChild actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true resource = Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000000, lookup_type: ById(001de000-5110-4000-8000-000000000000) } result = Ok(()) saga_id = dd4f75ce-0539-4224-8fc6-8244063d4551 saga_name = project-create saga_node = ProjectCreateRecord
36222025-04-30T06:43:52.921ZTRACtest_utilization (SagaExecutor): authorize begin action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true resource = Database saga_id = dd4f75ce-0539-4224-8fc6-8244063d4551 saga_name = project-create saga_node = ProjectCreateRecord
36232025-04-30T06:43:52.921ZDEBGtest_utilization (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true roles = RoleSet { roles: {} } saga_id = dd4f75ce-0539-4224-8fc6-8244063d4551 saga_name = project-create saga_node = ProjectCreateRecord
36242025-04-30T06:43:52.922ZDEBGtest_utilization (SagaExecutor): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true resource = Database result = Ok(()) saga_id = dd4f75ce-0539-4224-8fc6-8244063d4551 saga_name = project-create saga_node = ProjectCreateRecord
36252025-04-30T06:43:52.946ZDEBGtest_utilization (SEC): saga log event new_state = N000 succeeded sec_id = ac55d8d9-a298-4534-898b-beee1c9e4e52
36262025-04-30T06:43:52.946ZDEBGtest_utilization (SecStore): recording saga event event_type = Succeeded(Array [Object {"key": String("473b39f9-8669-45b3-b3d7-793f7e10fc31"), "lookup_type": Object {"ByName": String("utilization-test-project")}, "parent": Object {"key": String("001de000-5110-4000-8000-000000000000"), "lookup_type": Object {"ById": String("001de000-5110-4000-8000-000000000000")}, "parent": Null}}, Object {"identity": Object {"description": String("a pier"), "id": String("473b39f9-8669-45b3-b3d7-793f7e10fc31"), "name": String("utilization-test-project"), "time_created": String("2025-04-30T06:43:52.898759Z"), "time_deleted": Null, "time_modified": String("2025-04-30T06:43:52.898759Z")}, "rcgen": Number(1), "silo_id": String("001de000-5110-4000-8000-000000000000")}]) node_id = 0 saga_id = dd4f75ce-0539-4224-8fc6-8244063d4551
36272025-04-30T06:43:52.965ZDEBGtest_utilization (SEC): saga log event new_state = N001 started sec_id = ac55d8d9-a298-4534-898b-beee1c9e4e52
36282025-04-30T06:43:52.965ZDEBGtest_utilization (SecStore): recording saga event event_type = Started node_id = 1 saga_id = dd4f75ce-0539-4224-8fc6-8244063d4551
36292025-04-30T06:43:52.972ZDEBGtest_utilization (SEC): saga log event new_state = N001 succeeded sec_id = ac55d8d9-a298-4534-898b-beee1c9e4e52
36302025-04-30T06:43:52.972ZDEBGtest_utilization (SecStore): recording saga event event_type = Succeeded(Object {"authz_project": Object {"key": String("473b39f9-8669-45b3-b3d7-793f7e10fc31"), "lookup_type": Object {"ByName": String("utilization-test-project")}, "parent": Object {"key": String("001de000-5110-4000-8000-000000000000"), "lookup_type": Object {"ById": String("001de000-5110-4000-8000-000000000000")}, "parent": Null}}, "serialized_authn": Object {"kind": Object {"Authenticated": Array [Object {"actor": Object {"SiloUser": Object {"silo_id": String("001de000-5110-4000-8000-000000000000"), "silo_user_id": String("001de000-05e4-4000-8000-000000004007")}}}, Object {"mapped_fleet_roles": Object {}}]}}, "vpc_create": Object {"description": String("Default VPC"), "dns_name": String("default"), "ipv6_prefix": String("fdb8:6580:39ee::/48"), "name": String("default")}}) node_id = 1 saga_id = dd4f75ce-0539-4224-8fc6-8244063d4551
36312025-04-30T06:43:52.978ZDEBGtest_utilization (SEC): saga log event new_state = N002 started sec_id = ac55d8d9-a298-4534-898b-beee1c9e4e52
36322025-04-30T06:43:52.978ZDEBGtest_utilization (SecStore): recording saga event event_type = Started node_id = 2 saga_id = dd4f75ce-0539-4224-8fc6-8244063d4551
36332025-04-30T06:43:52.982ZDEBGtest_utilization (SEC): saga log event new_state = N002 succeeded sec_id = ac55d8d9-a298-4534-898b-beee1c9e4e52
36342025-04-30T06:43:52.982ZDEBGtest_utilization (SecStore): recording saga event event_type = Succeeded(Null) node_id = 2 saga_id = dd4f75ce-0539-4224-8fc6-8244063d4551
36352025-04-30T06:43:52.987ZDEBGtest_utilization (SEC): saga log event new_state = N003 started sec_id = ac55d8d9-a298-4534-898b-beee1c9e4e52
36362025-04-30T06:43:52.987ZDEBGtest_utilization (SecStore): recording saga event event_type = Started node_id = 3 saga_id = dd4f75ce-0539-4224-8fc6-8244063d4551
36372025-04-30T06:43:52.995ZDEBGtest_utilization (SEC): saga log event new_state = N003 succeeded sec_id = ac55d8d9-a298-4534-898b-beee1c9e4e52
36382025-04-30T06:43:52.995ZDEBGtest_utilization (SecStore): recording saga event event_type = Succeeded(String("cda41110-774f-439b-b99b-f715ebb74e33")) node_id = 3 saga_id = dd4f75ce-0539-4224-8fc6-8244063d4551
36392025-04-30T06:43:53.000ZDEBGtest_utilization (SEC): saga log event new_state = N004 started sec_id = ac55d8d9-a298-4534-898b-beee1c9e4e52
36402025-04-30T06:43:53.002ZDEBGtest_utilization (SecStore): recording saga event event_type = Started node_id = 4 saga_id = dd4f75ce-0539-4224-8fc6-8244063d4551
36412025-04-30T06:43:53.006ZDEBGtest_utilization (SEC): saga log event new_state = N004 succeeded sec_id = ac55d8d9-a298-4534-898b-beee1c9e4e52
36422025-04-30T06:43:53.006ZDEBGtest_utilization (SecStore): recording saga event event_type = Succeeded(String("fd90c8f9-ab47-4a3c-b34d-890ffe0ee4c4")) node_id = 4 saga_id = dd4f75ce-0539-4224-8fc6-8244063d4551
36432025-04-30T06:43:53.011ZDEBGtest_utilization (SEC): saga log event new_state = N005 started sec_id = ac55d8d9-a298-4534-898b-beee1c9e4e52
36442025-04-30T06:43:53.016ZDEBGtest_utilization (SecStore): recording saga event event_type = Started node_id = 5 saga_id = dd4f75ce-0539-4224-8fc6-8244063d4551
36452025-04-30T06:43:53.029ZDEBGtest_utilization (SEC): saga log event new_state = N005 succeeded sec_id = ac55d8d9-a298-4534-898b-beee1c9e4e52
36462025-04-30T06:43:53.029ZDEBGtest_utilization (SecStore): recording saga event event_type = Succeeded(String("e6961c2a-0f21-4758-9be0-412909d5abd1")) node_id = 5 saga_id = dd4f75ce-0539-4224-8fc6-8244063d4551
36472025-04-30T06:43:53.037ZDEBGtest_utilization (SEC): saga log event new_state = N006 started sec_id = ac55d8d9-a298-4534-898b-beee1c9e4e52
36482025-04-30T06:43:53.037ZDEBGtest_utilization (SecStore): recording saga event event_type = Started node_id = 6 saga_id = dd4f75ce-0539-4224-8fc6-8244063d4551
36492025-04-30T06:43:53.055ZDEBGtest_utilization (SEC): saga log event new_state = N006 succeeded sec_id = ac55d8d9-a298-4534-898b-beee1c9e4e52
36502025-04-30T06:43:53.055ZDEBGtest_utilization (SecStore): recording saga event event_type = Succeeded(String("a37eafa9-b3db-4363-aa0e-f96c1ac20163")) node_id = 6 saga_id = dd4f75ce-0539-4224-8fc6-8244063d4551
36512025-04-30T06:43:53.063ZDEBGtest_utilization (SEC): saga log event new_state = N007 started sec_id = ac55d8d9-a298-4534-898b-beee1c9e4e52
36522025-04-30T06:43:53.063ZDEBGtest_utilization (SecStore): recording saga event event_type = Started node_id = 7 saga_id = dd4f75ce-0539-4224-8fc6-8244063d4551
36532025-04-30T06:43:53.068ZDEBGtest_utilization (SEC): saga log event new_state = N007 succeeded sec_id = ac55d8d9-a298-4534-898b-beee1c9e4e52
36542025-04-30T06:43:53.068ZDEBGtest_utilization (SecStore): recording saga event event_type = Succeeded(String("e0639477-adf0-4eba-bf8b-9e814214f7fb")) node_id = 7 saga_id = dd4f75ce-0539-4224-8fc6-8244063d4551
36552025-04-30T06:43:53.072ZDEBGtest_utilization (SEC): saga log event new_state = N008 started sec_id = ac55d8d9-a298-4534-898b-beee1c9e4e52
36562025-04-30T06:43:53.072ZDEBGtest_utilization (SecStore): recording saga event event_type = Started node_id = 8 saga_id = dd4f75ce-0539-4224-8fc6-8244063d4551
36572025-04-30T06:43:53.076ZDEBGtest_utilization (SEC): saga log event new_state = N008 succeeded sec_id = ac55d8d9-a298-4534-898b-beee1c9e4e52
36582025-04-30T06:43:53.076ZDEBGtest_utilization (SecStore): recording saga event event_type = Succeeded(String("be0543da-af63-4252-b6b2-ce2bffd4dd49")) node_id = 8 saga_id = dd4f75ce-0539-4224-8fc6-8244063d4551
36592025-04-30T06:43:53.079ZDEBGtest_utilization (SEC): saga log event new_state = N009 started sec_id = ac55d8d9-a298-4534-898b-beee1c9e4e52
36602025-04-30T06:43:53.079ZDEBGtest_utilization (SecStore): recording saga event event_type = Started node_id = 9 saga_id = dd4f75ce-0539-4224-8fc6-8244063d4551
36612025-04-30T06:43:53.083ZDEBGtest_utilization (SEC): saga log event new_state = N009 succeeded sec_id = ac55d8d9-a298-4534-898b-beee1c9e4e52
36622025-04-30T06:43:53.083ZDEBGtest_utilization (SecStore): recording saga event event_type = Succeeded(String("5567c57c-3b46-4ca0-968f-03632bf48df0")) node_id = 9 saga_id = dd4f75ce-0539-4224-8fc6-8244063d4551
36632025-04-30T06:43:53.090ZDEBGtest_utilization (SEC): saga log event new_state = N010 started sec_id = ac55d8d9-a298-4534-898b-beee1c9e4e52
36642025-04-30T06:43:53.090ZDEBGtest_utilization (SecStore): recording saga event event_type = Started node_id = 10 saga_id = dd4f75ce-0539-4224-8fc6-8244063d4551
36652025-04-30T06:43:53.094ZTRACtest_utilization (SagaExecutor): authorize begin action = CreateChild actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true resource = Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000000, lookup_type: ById(001de000-5110-4000-8000-000000000000) }, key: 473b39f9-8669-45b3-b3d7-793f7e10fc31, lookup_type: ByName("utilization-test-project") } saga_id = dd4f75ce-0539-4224-8fc6-8244063d4551 saga_name = project-create saga_node = VpcCreateVpc
36662025-04-30T06:43:53.094ZTRACtest_utilization (SagaExecutor): loading roles actor = Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. } actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true resource_id = 473b39f9-8669-45b3-b3d7-793f7e10fc31 resource_type = Project saga_id = dd4f75ce-0539-4224-8fc6-8244063d4551 saga_name = project-create saga_node = VpcCreateVpc
36672025-04-30T06:43:53.094ZTRACtest_utilization (SagaExecutor): authorize begin action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true resource = Database saga_id = dd4f75ce-0539-4224-8fc6-8244063d4551 saga_name = project-create saga_node = VpcCreateVpc
36682025-04-30T06:43:53.094ZDEBGtest_utilization (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true roles = RoleSet { roles: {} } saga_id = dd4f75ce-0539-4224-8fc6-8244063d4551 saga_name = project-create saga_node = VpcCreateVpc
36692025-04-30T06:43:53.095ZDEBGtest_utilization (SagaExecutor): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true resource = Database result = Ok(()) saga_id = dd4f75ce-0539-4224-8fc6-8244063d4551 saga_name = project-create saga_node = VpcCreateVpc
36702025-04-30T06:43:53.102ZTRACtest_utilization (SagaExecutor): loading roles actor = Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. } actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true resource_id = 001de000-5110-4000-8000-000000000000 resource_type = Silo saga_id = dd4f75ce-0539-4224-8fc6-8244063d4551 saga_name = project-create saga_node = VpcCreateVpc
36712025-04-30T06:43:53.102ZTRACtest_utilization (SagaExecutor): authorize begin action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true resource = Database saga_id = dd4f75ce-0539-4224-8fc6-8244063d4551 saga_name = project-create saga_node = VpcCreateVpc
36722025-04-30T06:43:53.102ZDEBGtest_utilization (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true roles = RoleSet { roles: {} } saga_id = dd4f75ce-0539-4224-8fc6-8244063d4551 saga_name = project-create saga_node = VpcCreateVpc
36732025-04-30T06:43:53.103ZDEBGtest_utilization (SagaExecutor): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true resource = Database result = Ok(()) saga_id = dd4f75ce-0539-4224-8fc6-8244063d4551 saga_name = project-create saga_node = VpcCreateVpc
36742025-04-30T06:43:53.108ZTRACtest_utilization (SagaExecutor): loading roles actor = Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. } actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet saga_id = dd4f75ce-0539-4224-8fc6-8244063d4551 saga_name = project-create saga_node = VpcCreateVpc
36752025-04-30T06:43:53.108ZTRACtest_utilization (SagaExecutor): authorize begin action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true resource = Database saga_id = dd4f75ce-0539-4224-8fc6-8244063d4551 saga_name = project-create saga_node = VpcCreateVpc
36762025-04-30T06:43:53.108ZDEBGtest_utilization (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true roles = RoleSet { roles: {} } saga_id = dd4f75ce-0539-4224-8fc6-8244063d4551 saga_name = project-create saga_node = VpcCreateVpc
36772025-04-30T06:43:53.109ZDEBGtest_utilization (SagaExecutor): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true resource = Database result = Ok(()) saga_id = dd4f75ce-0539-4224-8fc6-8244063d4551 saga_name = project-create saga_node = VpcCreateVpc
36782025-04-30T06:43:53.120ZDEBGtest_utilization (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin"), (Silo, 001de000-5110-4000-8000-000000000000, "admin")} } saga_id = dd4f75ce-0539-4224-8fc6-8244063d4551 saga_name = project-create saga_node = VpcCreateVpc
36792025-04-30T06:43:53.128ZDEBGtest_utilization (SagaExecutor): authorize result action = CreateChild actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true resource = Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000000, lookup_type: ById(001de000-5110-4000-8000-000000000000) }, key: 473b39f9-8669-45b3-b3d7-793f7e10fc31, lookup_type: ByName("utilization-test-project") } result = Ok(()) saga_id = dd4f75ce-0539-4224-8fc6-8244063d4551 saga_name = project-create saga_node = VpcCreateVpc
36802025-04-30T06:43:53.130ZTRACtest_utilization (SagaExecutor): authorize begin action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true resource = Database saga_id = dd4f75ce-0539-4224-8fc6-8244063d4551 saga_name = project-create saga_node = VpcCreateVpc
36812025-04-30T06:43:53.130ZDEBGtest_utilization (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true roles = RoleSet { roles: {} } saga_id = dd4f75ce-0539-4224-8fc6-8244063d4551 saga_name = project-create saga_node = VpcCreateVpc
36822025-04-30T06:43:53.132ZDEBGtest_utilization (SagaExecutor): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true resource = Database result = Ok(()) saga_id = dd4f75ce-0539-4224-8fc6-8244063d4551 saga_name = project-create saga_node = VpcCreateVpc
36832025-04-30T06:43:53.145ZDEBGtest_utilization (SEC): saga log event new_state = N010 succeeded sec_id = ac55d8d9-a298-4534-898b-beee1c9e4e52
36842025-04-30T06:43:53.147ZDEBGtest_utilization (SecStore): recording saga event event_type = Succeeded(Array [Object {"key": String("cda41110-774f-439b-b99b-f715ebb74e33"), "lookup_type": Object {"ByName": String("default")}, "parent": Object {"key": String("473b39f9-8669-45b3-b3d7-793f7e10fc31"), "lookup_type": Object {"ByName": String("utilization-test-project")}, "parent": Object {"key": String("001de000-5110-4000-8000-000000000000"), "lookup_type": Object {"ById": String("001de000-5110-4000-8000-000000000000")}, "parent": Null}}}, Object {"dns_name": String("default"), "firewall_gen": Number(1), "identity": Object {"description": String("Default VPC"), "id": String("cda41110-774f-439b-b99b-f715ebb74e33"), "name": String("default"), "time_created": String("2025-04-30T06:43:53.094070Z"), "time_deleted": Null, "time_modified": String("2025-04-30T06:43:53.094070Z")}, "ipv6_prefix": String("fdb8:6580:39ee::/48"), "project_id": String("473b39f9-8669-45b3-b3d7-793f7e10fc31"), "subnet_gen": Number(1), "system_router_id": String("fd90c8f9-ab47-4a3c-b34d-890ffe0ee4c4"), "vni": Number(11911937)}]) node_id = 10 saga_id = dd4f75ce-0539-4224-8fc6-8244063d4551
36852025-04-30T06:43:53.160ZDEBGtest_utilization (SEC): saga log event new_state = N011 started sec_id = ac55d8d9-a298-4534-898b-beee1c9e4e52
36862025-04-30T06:43:53.160ZDEBGtest_utilization (SecStore): recording saga event event_type = Started node_id = 11 saga_id = dd4f75ce-0539-4224-8fc6-8244063d4551
36872025-04-30T06:43:53.164ZTRACtest_utilization (SagaExecutor): authorize begin action = CreateChild actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000000, lookup_type: ById(001de000-5110-4000-8000-000000000000) }, key: 473b39f9-8669-45b3-b3d7-793f7e10fc31, lookup_type: ByName("utilization-test-project") }, key: cda41110-774f-439b-b99b-f715ebb74e33, lookup_type: ByName("default") } saga_id = dd4f75ce-0539-4224-8fc6-8244063d4551 saga_name = project-create saga_node = VpcCreateRouter
36882025-04-30T06:43:53.164ZTRACtest_utilization (SagaExecutor): loading roles actor = Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. } actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true resource_id = 473b39f9-8669-45b3-b3d7-793f7e10fc31 resource_type = Project saga_id = dd4f75ce-0539-4224-8fc6-8244063d4551 saga_name = project-create saga_node = VpcCreateRouter
36892025-04-30T06:43:53.164ZTRACtest_utilization (SagaExecutor): authorize begin action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true resource = Database saga_id = dd4f75ce-0539-4224-8fc6-8244063d4551 saga_name = project-create saga_node = VpcCreateRouter
36902025-04-30T06:43:53.164ZDEBGtest_utilization (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true roles = RoleSet { roles: {} } saga_id = dd4f75ce-0539-4224-8fc6-8244063d4551 saga_name = project-create saga_node = VpcCreateRouter
36912025-04-30T06:43:53.165ZDEBGtest_utilization (SagaExecutor): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true resource = Database result = Ok(()) saga_id = dd4f75ce-0539-4224-8fc6-8244063d4551 saga_name = project-create saga_node = VpcCreateRouter
36922025-04-30T06:43:53.168ZTRACtest_utilization (SagaExecutor): loading roles actor = Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. } actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true resource_id = 001de000-5110-4000-8000-000000000000 resource_type = Silo saga_id = dd4f75ce-0539-4224-8fc6-8244063d4551 saga_name = project-create saga_node = VpcCreateRouter
36932025-04-30T06:43:53.168ZTRACtest_utilization (SagaExecutor): authorize begin action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true resource = Database saga_id = dd4f75ce-0539-4224-8fc6-8244063d4551 saga_name = project-create saga_node = VpcCreateRouter
36942025-04-30T06:43:53.168ZDEBGtest_utilization (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true roles = RoleSet { roles: {} } saga_id = dd4f75ce-0539-4224-8fc6-8244063d4551 saga_name = project-create saga_node = VpcCreateRouter
36952025-04-30T06:43:53.169ZDEBGtest_utilization (SagaExecutor): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true resource = Database result = Ok(()) saga_id = dd4f75ce-0539-4224-8fc6-8244063d4551 saga_name = project-create saga_node = VpcCreateRouter
36962025-04-30T06:43:53.172ZTRACtest_utilization (SagaExecutor): loading roles actor = Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. } actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet saga_id = dd4f75ce-0539-4224-8fc6-8244063d4551 saga_name = project-create saga_node = VpcCreateRouter
36972025-04-30T06:43:53.173ZTRACtest_utilization (SagaExecutor): authorize begin action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true resource = Database saga_id = dd4f75ce-0539-4224-8fc6-8244063d4551 saga_name = project-create saga_node = VpcCreateRouter
36982025-04-30T06:43:53.173ZDEBGtest_utilization (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true roles = RoleSet { roles: {} } saga_id = dd4f75ce-0539-4224-8fc6-8244063d4551 saga_name = project-create saga_node = VpcCreateRouter
36992025-04-30T06:43:53.174ZDEBGtest_utilization (SagaExecutor): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true resource = Database result = Ok(()) saga_id = dd4f75ce-0539-4224-8fc6-8244063d4551 saga_name = project-create saga_node = VpcCreateRouter
37002025-04-30T06:43:53.177ZDEBGtest_utilization (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin"), (Silo, 001de000-5110-4000-8000-000000000000, "admin")} } saga_id = dd4f75ce-0539-4224-8fc6-8244063d4551 saga_name = project-create saga_node = VpcCreateRouter
37012025-04-30T06:43:53.184ZDEBGtest_utilization (SagaExecutor): authorize result action = CreateChild actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000000, lookup_type: ById(001de000-5110-4000-8000-000000000000) }, key: 473b39f9-8669-45b3-b3d7-793f7e10fc31, lookup_type: ByName("utilization-test-project") }, key: cda41110-774f-439b-b99b-f715ebb74e33, lookup_type: ByName("default") } result = Ok(()) saga_id = dd4f75ce-0539-4224-8fc6-8244063d4551 saga_name = project-create saga_node = VpcCreateRouter
37022025-04-30T06:43:53.185ZTRACtest_utilization (SagaExecutor): authorize begin action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true resource = Database saga_id = dd4f75ce-0539-4224-8fc6-8244063d4551 saga_name = project-create saga_node = VpcCreateRouter
37032025-04-30T06:43:53.185ZDEBGtest_utilization (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true roles = RoleSet { roles: {} } saga_id = dd4f75ce-0539-4224-8fc6-8244063d4551 saga_name = project-create saga_node = VpcCreateRouter
37042025-04-30T06:43:53.185ZDEBGtest_utilization (SagaExecutor): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true resource = Database result = Ok(()) saga_id = dd4f75ce-0539-4224-8fc6-8244063d4551 saga_name = project-create saga_node = VpcCreateRouter
37052025-04-30T06:43:53.189ZDEBGtest_utilization (SEC): saga log event new_state = N011 succeeded sec_id = ac55d8d9-a298-4534-898b-beee1c9e4e52
37062025-04-30T06:43:53.189ZDEBGtest_utilization (SecStore): recording saga event event_type = Succeeded(Object {"key": String("fd90c8f9-ab47-4a3c-b34d-890ffe0ee4c4"), "lookup_type": Object {"ById": String("fd90c8f9-ab47-4a3c-b34d-890ffe0ee4c4")}, "parent": Object {"key": String("cda41110-774f-439b-b99b-f715ebb74e33"), "lookup_type": Object {"ByName": String("default")}, "parent": Object {"key": String("473b39f9-8669-45b3-b3d7-793f7e10fc31"), "lookup_type": Object {"ByName": String("utilization-test-project")}, "parent": Object {"key": String("001de000-5110-4000-8000-000000000000"), "lookup_type": Object {"ById": String("001de000-5110-4000-8000-000000000000")}, "parent": Null}}}}) node_id = 11 saga_id = dd4f75ce-0539-4224-8fc6-8244063d4551
37072025-04-30T06:43:53.192ZDEBGtest_utilization (SEC): saga log event new_state = N012 started sec_id = ac55d8d9-a298-4534-898b-beee1c9e4e52
37082025-04-30T06:43:53.193ZDEBGtest_utilization (SecStore): recording saga event event_type = Started node_id = 12 saga_id = dd4f75ce-0539-4224-8fc6-8244063d4551
37092025-04-30T06:43:53.198ZTRACtest_utilization (SagaExecutor): authorize begin action = CreateChild actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true resource = VpcRouter { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000000, lookup_type: ById(001de000-5110-4000-8000-000000000000) }, key: 473b39f9-8669-45b3-b3d7-793f7e10fc31, lookup_type: ByName("utilization-test-project") }, key: cda41110-774f-439b-b99b-f715ebb74e33, lookup_type: ByName("default") }, key: fd90c8f9-ab47-4a3c-b34d-890ffe0ee4c4, lookup_type: ById(fd90c8f9-ab47-4a3c-b34d-890ffe0ee4c4) } saga_id = dd4f75ce-0539-4224-8fc6-8244063d4551 saga_name = project-create saga_node = VpcCreateV4Route
37102025-04-30T06:43:53.198ZTRACtest_utilization (SagaExecutor): loading roles actor = Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. } actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true resource_id = 473b39f9-8669-45b3-b3d7-793f7e10fc31 resource_type = Project saga_id = dd4f75ce-0539-4224-8fc6-8244063d4551 saga_name = project-create saga_node = VpcCreateV4Route
37112025-04-30T06:43:53.199ZTRACtest_utilization (SagaExecutor): authorize begin action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true resource = Database saga_id = dd4f75ce-0539-4224-8fc6-8244063d4551 saga_name = project-create saga_node = VpcCreateV4Route
37122025-04-30T06:43:53.199ZDEBGtest_utilization (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true roles = RoleSet { roles: {} } saga_id = dd4f75ce-0539-4224-8fc6-8244063d4551 saga_name = project-create saga_node = VpcCreateV4Route
37132025-04-30T06:43:53.202ZDEBGtest_utilization (SagaExecutor): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true resource = Database result = Ok(()) saga_id = dd4f75ce-0539-4224-8fc6-8244063d4551 saga_name = project-create saga_node = VpcCreateV4Route
37142025-04-30T06:43:53.207ZTRACtest_utilization (SagaExecutor): loading roles actor = Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. } actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true resource_id = 001de000-5110-4000-8000-000000000000 resource_type = Silo saga_id = dd4f75ce-0539-4224-8fc6-8244063d4551 saga_name = project-create saga_node = VpcCreateV4Route
37152025-04-30T06:43:53.207ZTRACtest_utilization (SagaExecutor): authorize begin action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true resource = Database saga_id = dd4f75ce-0539-4224-8fc6-8244063d4551 saga_name = project-create saga_node = VpcCreateV4Route
37162025-04-30T06:43:53.207ZDEBGtest_utilization (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true roles = RoleSet { roles: {} } saga_id = dd4f75ce-0539-4224-8fc6-8244063d4551 saga_name = project-create saga_node = VpcCreateV4Route
37172025-04-30T06:43:53.209ZDEBGtest_utilization (SagaExecutor): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true resource = Database result = Ok(()) saga_id = dd4f75ce-0539-4224-8fc6-8244063d4551 saga_name = project-create saga_node = VpcCreateV4Route
37182025-04-30T06:43:53.213ZTRACtest_utilization (SagaExecutor): loading roles actor = Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. } actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet saga_id = dd4f75ce-0539-4224-8fc6-8244063d4551 saga_name = project-create saga_node = VpcCreateV4Route
37192025-04-30T06:43:53.214ZTRACtest_utilization (SagaExecutor): authorize begin action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true resource = Database saga_id = dd4f75ce-0539-4224-8fc6-8244063d4551 saga_name = project-create saga_node = VpcCreateV4Route
37202025-04-30T06:43:53.214ZDEBGtest_utilization (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true roles = RoleSet { roles: {} } saga_id = dd4f75ce-0539-4224-8fc6-8244063d4551 saga_name = project-create saga_node = VpcCreateV4Route
37212025-04-30T06:43:53.215ZDEBGtest_utilization (SagaExecutor): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true resource = Database result = Ok(()) saga_id = dd4f75ce-0539-4224-8fc6-8244063d4551 saga_name = project-create saga_node = VpcCreateV4Route
37222025-04-30T06:43:53.220ZDEBGtest_utilization (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin"), (Silo, 001de000-5110-4000-8000-000000000000, "admin")} } saga_id = dd4f75ce-0539-4224-8fc6-8244063d4551 saga_name = project-create saga_node = VpcCreateV4Route
37232025-04-30T06:43:53.227ZDEBGtest_utilization (SagaExecutor): authorize result action = CreateChild actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true resource = VpcRouter { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000000, lookup_type: ById(001de000-5110-4000-8000-000000000000) }, key: 473b39f9-8669-45b3-b3d7-793f7e10fc31, lookup_type: ByName("utilization-test-project") }, key: cda41110-774f-439b-b99b-f715ebb74e33, lookup_type: ByName("default") }, key: fd90c8f9-ab47-4a3c-b34d-890ffe0ee4c4, lookup_type: ById(fd90c8f9-ab47-4a3c-b34d-890ffe0ee4c4) } result = Ok(()) saga_id = dd4f75ce-0539-4224-8fc6-8244063d4551 saga_name = project-create saga_node = VpcCreateV4Route
37242025-04-30T06:43:53.228ZTRACtest_utilization (SagaExecutor): authorize begin action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true resource = Database saga_id = dd4f75ce-0539-4224-8fc6-8244063d4551 saga_name = project-create saga_node = VpcCreateV4Route
37252025-04-30T06:43:53.228ZDEBGtest_utilization (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true roles = RoleSet { roles: {} } saga_id = dd4f75ce-0539-4224-8fc6-8244063d4551 saga_name = project-create saga_node = VpcCreateV4Route
37262025-04-30T06:43:53.228ZDEBGtest_utilization (SagaExecutor): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true resource = Database result = Ok(()) saga_id = dd4f75ce-0539-4224-8fc6-8244063d4551 saga_name = project-create saga_node = VpcCreateV4Route
37272025-04-30T06:43:53.238ZDEBGtest_utilization (SEC): saga log event new_state = N012 succeeded sec_id = ac55d8d9-a298-4534-898b-beee1c9e4e52
37282025-04-30T06:43:53.239ZDEBGtest_utilization (SecStore): recording saga event event_type = Succeeded(Null) node_id = 12 saga_id = dd4f75ce-0539-4224-8fc6-8244063d4551
37292025-04-30T06:43:53.242ZDEBGtest_utilization (SEC): saga log event new_state = N013 started sec_id = ac55d8d9-a298-4534-898b-beee1c9e4e52
37302025-04-30T06:43:53.242ZDEBGtest_utilization (SecStore): recording saga event event_type = Started node_id = 13 saga_id = dd4f75ce-0539-4224-8fc6-8244063d4551
37312025-04-30T06:43:53.262ZTRACtest_utilization (SagaExecutor): authorize begin action = CreateChild actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true resource = VpcRouter { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000000, lookup_type: ById(001de000-5110-4000-8000-000000000000) }, key: 473b39f9-8669-45b3-b3d7-793f7e10fc31, lookup_type: ByName("utilization-test-project") }, key: cda41110-774f-439b-b99b-f715ebb74e33, lookup_type: ByName("default") }, key: fd90c8f9-ab47-4a3c-b34d-890ffe0ee4c4, lookup_type: ById(fd90c8f9-ab47-4a3c-b34d-890ffe0ee4c4) } saga_id = dd4f75ce-0539-4224-8fc6-8244063d4551 saga_name = project-create saga_node = VpcCreateV6Route
37322025-04-30T06:43:53.262ZTRACtest_utilization (SagaExecutor): loading roles actor = Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. } actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true resource_id = 473b39f9-8669-45b3-b3d7-793f7e10fc31 resource_type = Project saga_id = dd4f75ce-0539-4224-8fc6-8244063d4551 saga_name = project-create saga_node = VpcCreateV6Route
37332025-04-30T06:43:53.262ZTRACtest_utilization (SagaExecutor): authorize begin action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true resource = Database saga_id = dd4f75ce-0539-4224-8fc6-8244063d4551 saga_name = project-create saga_node = VpcCreateV6Route
37342025-04-30T06:43:53.262ZDEBGtest_utilization (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true roles = RoleSet { roles: {} } saga_id = dd4f75ce-0539-4224-8fc6-8244063d4551 saga_name = project-create saga_node = VpcCreateV6Route
37352025-04-30T06:43:53.263ZDEBGtest_utilization (SagaExecutor): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true resource = Database result = Ok(()) saga_id = dd4f75ce-0539-4224-8fc6-8244063d4551 saga_name = project-create saga_node = VpcCreateV6Route
37362025-04-30T06:43:53.268ZTRACtest_utilization (SagaExecutor): loading roles actor = Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. } actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true resource_id = 001de000-5110-4000-8000-000000000000 resource_type = Silo saga_id = dd4f75ce-0539-4224-8fc6-8244063d4551 saga_name = project-create saga_node = VpcCreateV6Route
37372025-04-30T06:43:53.269ZTRACtest_utilization (SagaExecutor): authorize begin action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true resource = Database saga_id = dd4f75ce-0539-4224-8fc6-8244063d4551 saga_name = project-create saga_node = VpcCreateV6Route
37382025-04-30T06:43:53.269ZDEBGtest_utilization (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true roles = RoleSet { roles: {} } saga_id = dd4f75ce-0539-4224-8fc6-8244063d4551 saga_name = project-create saga_node = VpcCreateV6Route
37392025-04-30T06:43:53.269ZDEBGtest_utilization (SagaExecutor): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true resource = Database result = Ok(()) saga_id = dd4f75ce-0539-4224-8fc6-8244063d4551 saga_name = project-create saga_node = VpcCreateV6Route
37402025-04-30T06:43:53.278ZTRACtest_utilization (SagaExecutor): loading roles actor = Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. } actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet saga_id = dd4f75ce-0539-4224-8fc6-8244063d4551 saga_name = project-create saga_node = VpcCreateV6Route
37412025-04-30T06:43:53.279ZTRACtest_utilization (SagaExecutor): authorize begin action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true resource = Database saga_id = dd4f75ce-0539-4224-8fc6-8244063d4551 saga_name = project-create saga_node = VpcCreateV6Route
37422025-04-30T06:43:53.279ZDEBGtest_utilization (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true roles = RoleSet { roles: {} } saga_id = dd4f75ce-0539-4224-8fc6-8244063d4551 saga_name = project-create saga_node = VpcCreateV6Route
37432025-04-30T06:43:53.279ZDEBGtest_utilization (SagaExecutor): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true resource = Database result = Ok(()) saga_id = dd4f75ce-0539-4224-8fc6-8244063d4551 saga_name = project-create saga_node = VpcCreateV6Route
37442025-04-30T06:43:53.285ZDEBGtest_utilization (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin"), (Silo, 001de000-5110-4000-8000-000000000000, "admin")} } saga_id = dd4f75ce-0539-4224-8fc6-8244063d4551 saga_name = project-create saga_node = VpcCreateV6Route
37452025-04-30T06:43:53.292ZDEBGtest_utilization (SagaExecutor): authorize result action = CreateChild actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true resource = VpcRouter { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000000, lookup_type: ById(001de000-5110-4000-8000-000000000000) }, key: 473b39f9-8669-45b3-b3d7-793f7e10fc31, lookup_type: ByName("utilization-test-project") }, key: cda41110-774f-439b-b99b-f715ebb74e33, lookup_type: ByName("default") }, key: fd90c8f9-ab47-4a3c-b34d-890ffe0ee4c4, lookup_type: ById(fd90c8f9-ab47-4a3c-b34d-890ffe0ee4c4) } result = Ok(()) saga_id = dd4f75ce-0539-4224-8fc6-8244063d4551 saga_name = project-create saga_node = VpcCreateV6Route
37462025-04-30T06:43:53.292ZTRACtest_utilization (SagaExecutor): authorize begin action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true resource = Database saga_id = dd4f75ce-0539-4224-8fc6-8244063d4551 saga_name = project-create saga_node = VpcCreateV6Route
37472025-04-30T06:43:53.292ZDEBGtest_utilization (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true roles = RoleSet { roles: {} } saga_id = dd4f75ce-0539-4224-8fc6-8244063d4551 saga_name = project-create saga_node = VpcCreateV6Route
37482025-04-30T06:43:53.292ZDEBGtest_utilization (SagaExecutor): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true resource = Database result = Ok(()) saga_id = dd4f75ce-0539-4224-8fc6-8244063d4551 saga_name = project-create saga_node = VpcCreateV6Route
37492025-04-30T06:43:53.300ZDEBGtest_utilization (SEC): saga log event new_state = N013 succeeded sec_id = ac55d8d9-a298-4534-898b-beee1c9e4e52
37502025-04-30T06:43:53.300ZDEBGtest_utilization (SecStore): recording saga event event_type = Succeeded(Null) node_id = 13 saga_id = dd4f75ce-0539-4224-8fc6-8244063d4551
37512025-04-30T06:43:53.311ZDEBGtest_utilization (SEC): saga log event new_state = N014 started sec_id = ac55d8d9-a298-4534-898b-beee1c9e4e52
37522025-04-30T06:43:53.311ZDEBGtest_utilization (SecStore): recording saga event event_type = Started node_id = 14 saga_id = dd4f75ce-0539-4224-8fc6-8244063d4551
37532025-04-30T06:43:53.314ZTRACtest_utilization (SagaExecutor): authorize begin action = CreateChild actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000000, lookup_type: ById(001de000-5110-4000-8000-000000000000) }, key: 473b39f9-8669-45b3-b3d7-793f7e10fc31, lookup_type: ByName("utilization-test-project") }, key: cda41110-774f-439b-b99b-f715ebb74e33, lookup_type: ByName("default") } saga_id = dd4f75ce-0539-4224-8fc6-8244063d4551 saga_name = project-create saga_node = VpcCreateSubnet
37542025-04-30T06:43:53.314ZTRACtest_utilization (SagaExecutor): loading roles actor = Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. } actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true resource_id = 473b39f9-8669-45b3-b3d7-793f7e10fc31 resource_type = Project saga_id = dd4f75ce-0539-4224-8fc6-8244063d4551 saga_name = project-create saga_node = VpcCreateSubnet
37552025-04-30T06:43:53.315ZTRACtest_utilization (SagaExecutor): authorize begin action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true resource = Database saga_id = dd4f75ce-0539-4224-8fc6-8244063d4551 saga_name = project-create saga_node = VpcCreateSubnet
37562025-04-30T06:43:53.315ZDEBGtest_utilization (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true roles = RoleSet { roles: {} } saga_id = dd4f75ce-0539-4224-8fc6-8244063d4551 saga_name = project-create saga_node = VpcCreateSubnet
37572025-04-30T06:43:53.315ZDEBGtest_utilization (SagaExecutor): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true resource = Database result = Ok(()) saga_id = dd4f75ce-0539-4224-8fc6-8244063d4551 saga_name = project-create saga_node = VpcCreateSubnet
37582025-04-30T06:43:53.320ZTRACtest_utilization (SagaExecutor): loading roles actor = Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. } actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true resource_id = 001de000-5110-4000-8000-000000000000 resource_type = Silo saga_id = dd4f75ce-0539-4224-8fc6-8244063d4551 saga_name = project-create saga_node = VpcCreateSubnet
37592025-04-30T06:43:53.320ZTRACtest_utilization (SagaExecutor): authorize begin action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true resource = Database saga_id = dd4f75ce-0539-4224-8fc6-8244063d4551 saga_name = project-create saga_node = VpcCreateSubnet
37602025-04-30T06:43:53.320ZDEBGtest_utilization (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true roles = RoleSet { roles: {} } saga_id = dd4f75ce-0539-4224-8fc6-8244063d4551 saga_name = project-create saga_node = VpcCreateSubnet
37612025-04-30T06:43:53.321ZDEBGtest_utilization (SagaExecutor): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true resource = Database result = Ok(()) saga_id = dd4f75ce-0539-4224-8fc6-8244063d4551 saga_name = project-create saga_node = VpcCreateSubnet
37622025-04-30T06:43:53.325ZTRACtest_utilization (SagaExecutor): loading roles actor = Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. } actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet saga_id = dd4f75ce-0539-4224-8fc6-8244063d4551 saga_name = project-create saga_node = VpcCreateSubnet
37632025-04-30T06:43:53.325ZTRACtest_utilization (SagaExecutor): authorize begin action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true resource = Database saga_id = dd4f75ce-0539-4224-8fc6-8244063d4551 saga_name = project-create saga_node = VpcCreateSubnet
37642025-04-30T06:43:53.325ZDEBGtest_utilization (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true roles = RoleSet { roles: {} } saga_id = dd4f75ce-0539-4224-8fc6-8244063d4551 saga_name = project-create saga_node = VpcCreateSubnet
37652025-04-30T06:43:53.326ZDEBGtest_utilization (SagaExecutor): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true resource = Database result = Ok(()) saga_id = dd4f75ce-0539-4224-8fc6-8244063d4551 saga_name = project-create saga_node = VpcCreateSubnet
37662025-04-30T06:43:53.328ZDEBGtest_utilization (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin"), (Silo, 001de000-5110-4000-8000-000000000000, "admin")} } saga_id = dd4f75ce-0539-4224-8fc6-8244063d4551 saga_name = project-create saga_node = VpcCreateSubnet
37672025-04-30T06:43:53.334ZDEBGtest_utilization (SagaExecutor): authorize result action = CreateChild actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000000, lookup_type: ById(001de000-5110-4000-8000-000000000000) }, key: 473b39f9-8669-45b3-b3d7-793f7e10fc31, lookup_type: ByName("utilization-test-project") }, key: cda41110-774f-439b-b99b-f715ebb74e33, lookup_type: ByName("default") } result = Ok(()) saga_id = dd4f75ce-0539-4224-8fc6-8244063d4551 saga_name = project-create saga_node = VpcCreateSubnet
37682025-04-30T06:43:53.338ZDEBGtest_utilization (SEC): saga log event new_state = N014 succeeded sec_id = ac55d8d9-a298-4534-898b-beee1c9e4e52
37692025-04-30T06:43:53.338ZDEBGtest_utilization (SecStore): recording saga event event_type = Succeeded(Array [Object {"key": String("be0543da-af63-4252-b6b2-ce2bffd4dd49"), "lookup_type": Object {"ById": String("be0543da-af63-4252-b6b2-ce2bffd4dd49")}, "parent": Object {"key": String("cda41110-774f-439b-b99b-f715ebb74e33"), "lookup_type": Object {"ByName": String("default")}, "parent": Object {"key": String("473b39f9-8669-45b3-b3d7-793f7e10fc31"), "lookup_type": Object {"ByName": String("utilization-test-project")}, "parent": Object {"key": String("001de000-5110-4000-8000-000000000000"), "lookup_type": Object {"ById": String("001de000-5110-4000-8000-000000000000")}, "parent": Null}}}}, Object {"custom_router_id": Null, "identity": Object {"description": String("The default subnet for default"), "id": String("be0543da-af63-4252-b6b2-ce2bffd4dd49"), "name": String("default"), "time_created": String("2025-04-30T06:43:53.314797Z"), "time_deleted": Null, "time_modified": String("2025-04-30T06:43:53.314797Z")}, "ipv4_block": String("172.30.0.0/22"), "ipv6_block": String("fdb8:6580:39ee::/64"), "rcgen": Number(1), "vpc_id": String("cda41110-774f-439b-b99b-f715ebb74e33")}]) node_id = 14 saga_id = dd4f75ce-0539-4224-8fc6-8244063d4551
37702025-04-30T06:43:53.345ZDEBGtest_utilization (SEC): saga log event new_state = N015 started sec_id = ac55d8d9-a298-4534-898b-beee1c9e4e52
37712025-04-30T06:43:53.345ZDEBGtest_utilization (SecStore): recording saga event event_type = Started node_id = 15 saga_id = dd4f75ce-0539-4224-8fc6-8244063d4551
37722025-04-30T06:43:53.351ZTRACtest_utilization (SagaExecutor): authorize begin action = CreateChild actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true resource = VpcRouter { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000000, lookup_type: ById(001de000-5110-4000-8000-000000000000) }, key: 473b39f9-8669-45b3-b3d7-793f7e10fc31, lookup_type: ByName("utilization-test-project") }, key: cda41110-774f-439b-b99b-f715ebb74e33, lookup_type: ByName("default") }, key: fd90c8f9-ab47-4a3c-b34d-890ffe0ee4c4, lookup_type: ById(fd90c8f9-ab47-4a3c-b34d-890ffe0ee4c4) } saga_id = dd4f75ce-0539-4224-8fc6-8244063d4551 saga_name = project-create saga_node = VpcCreateSubnetRoute
37732025-04-30T06:43:53.351ZTRACtest_utilization (SagaExecutor): loading roles actor = Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. } actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true resource_id = 473b39f9-8669-45b3-b3d7-793f7e10fc31 resource_type = Project saga_id = dd4f75ce-0539-4224-8fc6-8244063d4551 saga_name = project-create saga_node = VpcCreateSubnetRoute
37742025-04-30T06:43:53.351ZTRACtest_utilization (SagaExecutor): authorize begin action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true resource = Database saga_id = dd4f75ce-0539-4224-8fc6-8244063d4551 saga_name = project-create saga_node = VpcCreateSubnetRoute
37752025-04-30T06:43:53.351ZDEBGtest_utilization (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true roles = RoleSet { roles: {} } saga_id = dd4f75ce-0539-4224-8fc6-8244063d4551 saga_name = project-create saga_node = VpcCreateSubnetRoute
37762025-04-30T06:43:53.352ZDEBGtest_utilization (SagaExecutor): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true resource = Database result = Ok(()) saga_id = dd4f75ce-0539-4224-8fc6-8244063d4551 saga_name = project-create saga_node = VpcCreateSubnetRoute
37772025-04-30T06:43:53.358ZTRACtest_utilization (results-sink): batch interval expired, but no samples to insert collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
37782025-04-30T06:43:53.358ZTRACtest_utilization (results-sink-cluster): batch interval expired, but no samples to insert collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
37792025-04-30T06:43:53.361ZTRACtest_utilization (SagaExecutor): loading roles actor = Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. } actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true resource_id = 001de000-5110-4000-8000-000000000000 resource_type = Silo saga_id = dd4f75ce-0539-4224-8fc6-8244063d4551 saga_name = project-create saga_node = VpcCreateSubnetRoute
37802025-04-30T06:43:53.361ZTRACtest_utilization (SagaExecutor): authorize begin action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true resource = Database saga_id = dd4f75ce-0539-4224-8fc6-8244063d4551 saga_name = project-create saga_node = VpcCreateSubnetRoute
37812025-04-30T06:43:53.361ZDEBGtest_utilization (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true roles = RoleSet { roles: {} } saga_id = dd4f75ce-0539-4224-8fc6-8244063d4551 saga_name = project-create saga_node = VpcCreateSubnetRoute
37822025-04-30T06:43:53.362ZDEBGtest_utilization (SagaExecutor): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true resource = Database result = Ok(()) saga_id = dd4f75ce-0539-4224-8fc6-8244063d4551 saga_name = project-create saga_node = VpcCreateSubnetRoute
37832025-04-30T06:43:53.369ZTRACtest_utilization (SagaExecutor): loading roles actor = Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. } actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet saga_id = dd4f75ce-0539-4224-8fc6-8244063d4551 saga_name = project-create saga_node = VpcCreateSubnetRoute
37842025-04-30T06:43:53.369ZTRACtest_utilization (SagaExecutor): authorize begin action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true resource = Database saga_id = dd4f75ce-0539-4224-8fc6-8244063d4551 saga_name = project-create saga_node = VpcCreateSubnetRoute
37852025-04-30T06:43:53.369ZDEBGtest_utilization (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true roles = RoleSet { roles: {} } saga_id = dd4f75ce-0539-4224-8fc6-8244063d4551 saga_name = project-create saga_node = VpcCreateSubnetRoute
37862025-04-30T06:43:53.369ZDEBGtest_utilization (SagaExecutor): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true resource = Database result = Ok(()) saga_id = dd4f75ce-0539-4224-8fc6-8244063d4551 saga_name = project-create saga_node = VpcCreateSubnetRoute
37872025-04-30T06:43:53.374ZDEBGtest_utilization (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin"), (Silo, 001de000-5110-4000-8000-000000000000, "admin")} } saga_id = dd4f75ce-0539-4224-8fc6-8244063d4551 saga_name = project-create saga_node = VpcCreateSubnetRoute
37882025-04-30T06:43:53.383ZDEBGtest_utilization (SagaExecutor): authorize result action = CreateChild actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true resource = VpcRouter { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000000, lookup_type: ById(001de000-5110-4000-8000-000000000000) }, key: 473b39f9-8669-45b3-b3d7-793f7e10fc31, lookup_type: ByName("utilization-test-project") }, key: cda41110-774f-439b-b99b-f715ebb74e33, lookup_type: ByName("default") }, key: fd90c8f9-ab47-4a3c-b34d-890ffe0ee4c4, lookup_type: ById(fd90c8f9-ab47-4a3c-b34d-890ffe0ee4c4) } result = Ok(()) saga_id = dd4f75ce-0539-4224-8fc6-8244063d4551 saga_name = project-create saga_node = VpcCreateSubnetRoute
37892025-04-30T06:43:53.383ZTRACtest_utilization (SagaExecutor): authorize begin action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true resource = Database saga_id = dd4f75ce-0539-4224-8fc6-8244063d4551 saga_name = project-create saga_node = VpcCreateSubnetRoute
37902025-04-30T06:43:53.383ZDEBGtest_utilization (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true roles = RoleSet { roles: {} } saga_id = dd4f75ce-0539-4224-8fc6-8244063d4551 saga_name = project-create saga_node = VpcCreateSubnetRoute
37912025-04-30T06:43:53.384ZDEBGtest_utilization (SagaExecutor): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true resource = Database result = Ok(()) saga_id = dd4f75ce-0539-4224-8fc6-8244063d4551 saga_name = project-create saga_node = VpcCreateSubnetRoute
37922025-04-30T06:43:53.392ZDEBGtest_utilization (SEC): saga log event new_state = N015 succeeded sec_id = ac55d8d9-a298-4534-898b-beee1c9e4e52
37932025-04-30T06:43:53.392ZDEBGtest_utilization (SecStore): recording saga event event_type = Succeeded(Object {"key": String("e0639477-adf0-4eba-bf8b-9e814214f7fb"), "lookup_type": Object {"ById": String("be0543da-af63-4252-b6b2-ce2bffd4dd49")}, "parent": Object {"key": String("fd90c8f9-ab47-4a3c-b34d-890ffe0ee4c4"), "lookup_type": Object {"ById": String("fd90c8f9-ab47-4a3c-b34d-890ffe0ee4c4")}, "parent": Object {"key": String("cda41110-774f-439b-b99b-f715ebb74e33"), "lookup_type": Object {"ByName": String("default")}, "parent": Object {"key": String("473b39f9-8669-45b3-b3d7-793f7e10fc31"), "lookup_type": Object {"ByName": String("utilization-test-project")}, "parent": Object {"key": String("001de000-5110-4000-8000-000000000000"), "lookup_type": Object {"ById": String("001de000-5110-4000-8000-000000000000")}, "parent": Null}}}}}) node_id = 15 saga_id = dd4f75ce-0539-4224-8fc6-8244063d4551
37942025-04-30T06:43:53.397ZDEBGtest_utilization (SEC): saga log event new_state = N016 started sec_id = ac55d8d9-a298-4534-898b-beee1c9e4e52
37952025-04-30T06:43:53.397ZDEBGtest_utilization (SecStore): recording saga event event_type = Started node_id = 16 saga_id = dd4f75ce-0539-4224-8fc6-8244063d4551
37962025-04-30T06:43:53.402ZDEBGtest_utilization (nexus): default firewall rules for vpc default rules = [VpcFirewallRule { identity: VpcFirewallRuleIdentity { id: 168827d1-2d45-47a9-a589-52bf9bfc93aa, name: Name(Name("allow-internal-inbound")), description: "allow inbound traffic to all instances within the VPC if originated within the VPC", time_created: 2025-04-30T06:43:53.401895291Z, time_modified: 2025-04-30T06:43:53.401895291Z, time_deleted: None }, vpc_id: cda41110-774f-439b-b99b-f715ebb74e33, status: VpcFirewallRuleStatus(Enabled), direction: VpcFirewallRuleDirection(Inbound), targets: [VpcFirewallRuleTarget(Vpc(Name("default")))], filter_hosts: Some([VpcFirewallRuleHostFilter(Vpc(Name("default")))]), filter_ports: None, filter_protocols: None, action: VpcFirewallRuleAction(Allow), priority: VpcFirewallRulePriority(VpcFirewallRulePriority(65534)) }, VpcFirewallRule { identity: VpcFirewallRuleIdentity { id: cab9a6af-a8a8-4fa2-80b1-0e0b67193155, name: Name(Name("allow-ssh")), description: "allow inbound TCP connections on port 22 from anywhere", time_created: 2025-04-30T06:43:53.401911463Z, time_modified: 2025-04-30T06:43:53.401911463Z, time_deleted: None }, vpc_id: cda41110-774f-439b-b99b-f715ebb74e33, status: VpcFirewallRuleStatus(Enabled), direction: VpcFirewallRuleDirection(Inbound), targets: [VpcFirewallRuleTarget(Vpc(Name("default")))], filter_hosts: None, filter_ports: Some([L4PortRange(L4PortRange { first: L4Port(22), last: L4Port(22) })]), filter_protocols: Some([VpcFirewallRuleProtocol(Tcp)]), action: VpcFirewallRuleAction(Allow), priority: VpcFirewallRulePriority(VpcFirewallRulePriority(65534)) }, VpcFirewallRule { identity: VpcFirewallRuleIdentity { id: 540073f5-235b-41db-9c1a-2d426500061e, name: Name(Name("allow-icmp")), description: "allow inbound ICMP traffic from anywhere", time_created: 2025-04-30T06:43:53.401918397Z, time_modified: 2025-04-30T06:43:53.401918397Z, time_deleted: None }, vpc_id: cda41110-774f-439b-b99b-f715ebb74e33, status: VpcFirewallRuleStatus(Enabled), direction: VpcFirewallRuleDirection(Inbound), targets: [VpcFirewallRuleTarget(Vpc(Name("default")))], filter_hosts: None, filter_ports: None, filter_protocols: Some([VpcFirewallRuleProtocol(Icmp)]), action: VpcFirewallRuleAction(Allow), priority: VpcFirewallRulePriority(VpcFirewallRulePriority(65534)) }]
37972025-04-30T06:43:53.402ZTRACtest_utilization (SagaExecutor): authorize begin action = Modify actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000000, lookup_type: ById(001de000-5110-4000-8000-000000000000) }, key: 473b39f9-8669-45b3-b3d7-793f7e10fc31, lookup_type: ByName("utilization-test-project") }, key: cda41110-774f-439b-b99b-f715ebb74e33, lookup_type: ByName("default") } saga_id = dd4f75ce-0539-4224-8fc6-8244063d4551 saga_name = project-create saga_node = VpcUpdateFirewall
37982025-04-30T06:43:53.402ZTRACtest_utilization (SagaExecutor): loading roles actor = Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. } actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true resource_id = 473b39f9-8669-45b3-b3d7-793f7e10fc31 resource_type = Project saga_id = dd4f75ce-0539-4224-8fc6-8244063d4551 saga_name = project-create saga_node = VpcUpdateFirewall
37992025-04-30T06:43:53.403ZTRACtest_utilization (SagaExecutor): authorize begin action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true resource = Database saga_id = dd4f75ce-0539-4224-8fc6-8244063d4551 saga_name = project-create saga_node = VpcUpdateFirewall
38002025-04-30T06:43:53.403ZDEBGtest_utilization (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true roles = RoleSet { roles: {} } saga_id = dd4f75ce-0539-4224-8fc6-8244063d4551 saga_name = project-create saga_node = VpcUpdateFirewall
38012025-04-30T06:43:53.404ZDEBGtest_utilization (SagaExecutor): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true resource = Database result = Ok(()) saga_id = dd4f75ce-0539-4224-8fc6-8244063d4551 saga_name = project-create saga_node = VpcUpdateFirewall
38022025-04-30T06:43:53.407ZTRACtest_utilization (SagaExecutor): loading roles actor = Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. } actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true resource_id = 001de000-5110-4000-8000-000000000000 resource_type = Silo saga_id = dd4f75ce-0539-4224-8fc6-8244063d4551 saga_name = project-create saga_node = VpcUpdateFirewall
38032025-04-30T06:43:53.407ZTRACtest_utilization (SagaExecutor): authorize begin action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true resource = Database saga_id = dd4f75ce-0539-4224-8fc6-8244063d4551 saga_name = project-create saga_node = VpcUpdateFirewall
38042025-04-30T06:43:53.407ZDEBGtest_utilization (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true roles = RoleSet { roles: {} } saga_id = dd4f75ce-0539-4224-8fc6-8244063d4551 saga_name = project-create saga_node = VpcUpdateFirewall
38052025-04-30T06:43:53.408ZDEBGtest_utilization (SagaExecutor): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true resource = Database result = Ok(()) saga_id = dd4f75ce-0539-4224-8fc6-8244063d4551 saga_name = project-create saga_node = VpcUpdateFirewall
38062025-04-30T06:43:53.413ZTRACtest_utilization (SagaExecutor): loading roles actor = Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. } actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet saga_id = dd4f75ce-0539-4224-8fc6-8244063d4551 saga_name = project-create saga_node = VpcUpdateFirewall
38072025-04-30T06:43:53.413ZTRACtest_utilization (SagaExecutor): authorize begin action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true resource = Database saga_id = dd4f75ce-0539-4224-8fc6-8244063d4551 saga_name = project-create saga_node = VpcUpdateFirewall
38082025-04-30T06:43:53.413ZDEBGtest_utilization (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true roles = RoleSet { roles: {} } saga_id = dd4f75ce-0539-4224-8fc6-8244063d4551 saga_name = project-create saga_node = VpcUpdateFirewall
38092025-04-30T06:43:53.414ZDEBGtest_utilization (SagaExecutor): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true resource = Database result = Ok(()) saga_id = dd4f75ce-0539-4224-8fc6-8244063d4551 saga_name = project-create saga_node = VpcUpdateFirewall
38102025-04-30T06:43:53.417ZDEBGtest_utilization (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin"), (Silo, 001de000-5110-4000-8000-000000000000, "admin")} } saga_id = dd4f75ce-0539-4224-8fc6-8244063d4551 saga_name = project-create saga_node = VpcUpdateFirewall
38112025-04-30T06:43:53.425ZDEBGtest_utilization (SagaExecutor): authorize result action = Modify actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000000, lookup_type: ById(001de000-5110-4000-8000-000000000000) }, key: 473b39f9-8669-45b3-b3d7-793f7e10fc31, lookup_type: ByName("utilization-test-project") }, key: cda41110-774f-439b-b99b-f715ebb74e33, lookup_type: ByName("default") } result = Ok(()) saga_id = dd4f75ce-0539-4224-8fc6-8244063d4551 saga_name = project-create saga_node = VpcUpdateFirewall
38122025-04-30T06:43:53.425ZTRACtest_utilization (SagaExecutor): authorize begin action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true resource = Database saga_id = dd4f75ce-0539-4224-8fc6-8244063d4551 saga_name = project-create saga_node = VpcUpdateFirewall
38132025-04-30T06:43:53.425ZDEBGtest_utilization (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true roles = RoleSet { roles: {} } saga_id = dd4f75ce-0539-4224-8fc6-8244063d4551 saga_name = project-create saga_node = VpcUpdateFirewall
38142025-04-30T06:43:53.427ZDEBGtest_utilization (SagaExecutor): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true resource = Database result = Ok(()) saga_id = dd4f75ce-0539-4224-8fc6-8244063d4551 saga_name = project-create saga_node = VpcUpdateFirewall
38152025-04-30T06:43:53.451ZDEBGtest_utilization (SEC): saga log event new_state = N016 succeeded sec_id = ac55d8d9-a298-4534-898b-beee1c9e4e52
38162025-04-30T06:43:53.451ZDEBGtest_utilization (SecStore): recording saga event event_type = Succeeded(Array [Object {"action": String("allow"), "direction": String("inbound"), "filter_hosts": Array [Object {"type": String("vpc"), "value": String("default")}], "filter_ports": Null, "filter_protocols": Null, "identity": Object {"description": String("allow inbound traffic to all instances within the VPC if originated within the VPC"), "id": String("168827d1-2d45-47a9-a589-52bf9bfc93aa"), "name": String("allow-internal-inbound"), "time_created": String("2025-04-30T06:43:53.401895291Z"), "time_deleted": Null, "time_modified": String("2025-04-30T06:43:53.401895291Z")}, "priority": Number(65534), "status": String("enabled"), "targets": Array [Object {"type": String("vpc"), "value": String("default")}], "vpc_id": String("cda41110-774f-439b-b99b-f715ebb74e33")}, Object {"action": String("allow"), "direction": String("inbound"), "filter_hosts": Null, "filter_ports": Array [String("22")], "filter_protocols": Array [String("TCP")], "identity": Object {"description": String("allow inbound TCP connections on port 22 from anywhere"), "id": String("cab9a6af-a8a8-4fa2-80b1-0e0b67193155"), "name": String("allow-ssh"), "time_created": String("2025-04-30T06:43:53.401911463Z"), "time_deleted": Null, "time_modified": String("2025-04-30T06:43:53.401911463Z")}, "priority": Number(65534), "status": String("enabled"), "targets": Array [Object {"type": String("vpc"), "value": String("default")}], "vpc_id": String("cda41110-774f-439b-b99b-f715ebb74e33")}, Object {"action": String("allow"), "direction": String("inbound"), "filter_hosts": Null, "filter_ports": Null, "filter_protocols": Array [String("ICMP")], "identity": Object {"description": String("allow inbound ICMP traffic from anywhere"), "id": String("540073f5-235b-41db-9c1a-2d426500061e"), "name": String("allow-icmp"), "time_created": String("2025-04-30T06:43:53.401918397Z"), "time_deleted": Null, "time_modified": String("2025-04-30T06:43:53.401918397Z")}, "priority": Number(65534), "status": String("enabled"), "targets": Array [Object {"type": String("vpc"), "value": String("default")}], "vpc_id": String("cda41110-774f-439b-b99b-f715ebb74e33")}]) node_id = 16 saga_id = dd4f75ce-0539-4224-8fc6-8244063d4551
38172025-04-30T06:43:53.459ZDEBGtest_utilization (SEC): saga log event new_state = N017 started sec_id = ac55d8d9-a298-4534-898b-beee1c9e4e52
38182025-04-30T06:43:53.459ZDEBGtest_utilization (SecStore): recording saga event event_type = Started node_id = 17 saga_id = dd4f75ce-0539-4224-8fc6-8244063d4551
38192025-04-30T06:43:53.463ZTRACtest_utilization (SagaExecutor): authorize begin action = CreateChild actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000000, lookup_type: ById(001de000-5110-4000-8000-000000000000) }, key: 473b39f9-8669-45b3-b3d7-793f7e10fc31, lookup_type: ByName("utilization-test-project") }, key: cda41110-774f-439b-b99b-f715ebb74e33, lookup_type: ByName("default") } saga_id = dd4f75ce-0539-4224-8fc6-8244063d4551 saga_name = project-create saga_node = VpcCreateGateway
38202025-04-30T06:43:53.463ZTRACtest_utilization (SagaExecutor): loading roles actor = Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. } actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true resource_id = 473b39f9-8669-45b3-b3d7-793f7e10fc31 resource_type = Project saga_id = dd4f75ce-0539-4224-8fc6-8244063d4551 saga_name = project-create saga_node = VpcCreateGateway
38212025-04-30T06:43:53.463ZTRACtest_utilization (SagaExecutor): authorize begin action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true resource = Database saga_id = dd4f75ce-0539-4224-8fc6-8244063d4551 saga_name = project-create saga_node = VpcCreateGateway
38222025-04-30T06:43:53.463ZDEBGtest_utilization (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true roles = RoleSet { roles: {} } saga_id = dd4f75ce-0539-4224-8fc6-8244063d4551 saga_name = project-create saga_node = VpcCreateGateway
38232025-04-30T06:43:53.463ZDEBGtest_utilization (SagaExecutor): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true resource = Database result = Ok(()) saga_id = dd4f75ce-0539-4224-8fc6-8244063d4551 saga_name = project-create saga_node = VpcCreateGateway
38242025-04-30T06:43:53.466ZTRACtest_utilization (SagaExecutor): loading roles actor = Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. } actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true resource_id = 001de000-5110-4000-8000-000000000000 resource_type = Silo saga_id = dd4f75ce-0539-4224-8fc6-8244063d4551 saga_name = project-create saga_node = VpcCreateGateway
38252025-04-30T06:43:53.466ZTRACtest_utilization (SagaExecutor): authorize begin action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true resource = Database saga_id = dd4f75ce-0539-4224-8fc6-8244063d4551 saga_name = project-create saga_node = VpcCreateGateway
38262025-04-30T06:43:53.466ZDEBGtest_utilization (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true roles = RoleSet { roles: {} } saga_id = dd4f75ce-0539-4224-8fc6-8244063d4551 saga_name = project-create saga_node = VpcCreateGateway
38272025-04-30T06:43:53.466ZDEBGtest_utilization (SagaExecutor): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true resource = Database result = Ok(()) saga_id = dd4f75ce-0539-4224-8fc6-8244063d4551 saga_name = project-create saga_node = VpcCreateGateway
38282025-04-30T06:43:53.469ZTRACtest_utilization (SagaExecutor): loading roles actor = Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. } actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet saga_id = dd4f75ce-0539-4224-8fc6-8244063d4551 saga_name = project-create saga_node = VpcCreateGateway
38292025-04-30T06:43:53.469ZTRACtest_utilization (SagaExecutor): authorize begin action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true resource = Database saga_id = dd4f75ce-0539-4224-8fc6-8244063d4551 saga_name = project-create saga_node = VpcCreateGateway
38302025-04-30T06:43:53.469ZDEBGtest_utilization (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true roles = RoleSet { roles: {} } saga_id = dd4f75ce-0539-4224-8fc6-8244063d4551 saga_name = project-create saga_node = VpcCreateGateway
38312025-04-30T06:43:53.470ZDEBGtest_utilization (SagaExecutor): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true resource = Database result = Ok(()) saga_id = dd4f75ce-0539-4224-8fc6-8244063d4551 saga_name = project-create saga_node = VpcCreateGateway
38322025-04-30T06:43:53.473ZDEBGtest_utilization (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin"), (Silo, 001de000-5110-4000-8000-000000000000, "admin")} } saga_id = dd4f75ce-0539-4224-8fc6-8244063d4551 saga_name = project-create saga_node = VpcCreateGateway
38332025-04-30T06:43:53.478ZDEBGtest_utilization (SagaExecutor): authorize result action = CreateChild actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000000, lookup_type: ById(001de000-5110-4000-8000-000000000000) }, key: 473b39f9-8669-45b3-b3d7-793f7e10fc31, lookup_type: ByName("utilization-test-project") }, key: cda41110-774f-439b-b99b-f715ebb74e33, lookup_type: ByName("default") } result = Ok(()) saga_id = dd4f75ce-0539-4224-8fc6-8244063d4551 saga_name = project-create saga_node = VpcCreateGateway
38342025-04-30T06:43:53.479ZTRACtest_utilization (SagaExecutor): authorize begin action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true resource = Database saga_id = dd4f75ce-0539-4224-8fc6-8244063d4551 saga_name = project-create saga_node = VpcCreateGateway
38352025-04-30T06:43:53.479ZDEBGtest_utilization (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true roles = RoleSet { roles: {} } saga_id = dd4f75ce-0539-4224-8fc6-8244063d4551 saga_name = project-create saga_node = VpcCreateGateway
38362025-04-30T06:43:53.479ZDEBGtest_utilization (SagaExecutor): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true resource = Database result = Ok(()) saga_id = dd4f75ce-0539-4224-8fc6-8244063d4551 saga_name = project-create saga_node = VpcCreateGateway
38372025-04-30T06:43:53.481ZTRACtest_utilization (SagaExecutor): authorize begin action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true resource = Database saga_id = dd4f75ce-0539-4224-8fc6-8244063d4551 saga_name = project-create saga_node = VpcCreateGateway
38382025-04-30T06:43:53.481ZDEBGtest_utilization (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true roles = RoleSet { roles: {} } saga_id = dd4f75ce-0539-4224-8fc6-8244063d4551 saga_name = project-create saga_node = VpcCreateGateway
38392025-04-30T06:43:53.482ZDEBGtest_utilization (SagaExecutor): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true resource = Database result = Ok(()) saga_id = dd4f75ce-0539-4224-8fc6-8244063d4551 saga_name = project-create saga_node = VpcCreateGateway
38402025-04-30T06:43:53.492ZTRACtest_utilization (SagaExecutor): authorize begin action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true resource = Database saga_id = dd4f75ce-0539-4224-8fc6-8244063d4551 saga_name = project-create saga_node = VpcCreateGateway
38412025-04-30T06:43:53.492ZDEBGtest_utilization (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true roles = RoleSet { roles: {} } saga_id = dd4f75ce-0539-4224-8fc6-8244063d4551 saga_name = project-create saga_node = VpcCreateGateway
38422025-04-30T06:43:53.498ZDEBGtest_utilization (SagaExecutor): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true resource = Database result = Ok(()) saga_id = dd4f75ce-0539-4224-8fc6-8244063d4551 saga_name = project-create saga_node = VpcCreateGateway
38432025-04-30T06:43:53.504ZTRACtest_utilization (SagaExecutor): authorize begin action = CreateChild actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true resource = InternetGateway { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000000, lookup_type: ById(001de000-5110-4000-8000-000000000000) }, key: 473b39f9-8669-45b3-b3d7-793f7e10fc31, lookup_type: ByName("utilization-test-project") }, key: cda41110-774f-439b-b99b-f715ebb74e33, lookup_type: ByName("default") }, key: 5567c57c-3b46-4ca0-968f-03632bf48df0, lookup_type: ById(5567c57c-3b46-4ca0-968f-03632bf48df0) } saga_id = dd4f75ce-0539-4224-8fc6-8244063d4551 saga_name = project-create saga_node = VpcCreateGateway
38442025-04-30T06:43:53.504ZTRACtest_utilization (SagaExecutor): loading roles actor = Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. } actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true resource_id = 473b39f9-8669-45b3-b3d7-793f7e10fc31 resource_type = Project saga_id = dd4f75ce-0539-4224-8fc6-8244063d4551 saga_name = project-create saga_node = VpcCreateGateway
38452025-04-30T06:43:53.504ZTRACtest_utilization (SagaExecutor): authorize begin action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true resource = Database saga_id = dd4f75ce-0539-4224-8fc6-8244063d4551 saga_name = project-create saga_node = VpcCreateGateway
38462025-04-30T06:43:53.504ZDEBGtest_utilization (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true roles = RoleSet { roles: {} } saga_id = dd4f75ce-0539-4224-8fc6-8244063d4551 saga_name = project-create saga_node = VpcCreateGateway
38472025-04-30T06:43:53.505ZDEBGtest_utilization (SagaExecutor): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true resource = Database result = Ok(()) saga_id = dd4f75ce-0539-4224-8fc6-8244063d4551 saga_name = project-create saga_node = VpcCreateGateway
38482025-04-30T06:43:53.511ZTRACtest_utilization (SagaExecutor): loading roles actor = Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. } actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true resource_id = 001de000-5110-4000-8000-000000000000 resource_type = Silo saga_id = dd4f75ce-0539-4224-8fc6-8244063d4551 saga_name = project-create saga_node = VpcCreateGateway
38492025-04-30T06:43:53.511ZTRACtest_utilization (SagaExecutor): authorize begin action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true resource = Database saga_id = dd4f75ce-0539-4224-8fc6-8244063d4551 saga_name = project-create saga_node = VpcCreateGateway
38502025-04-30T06:43:53.511ZDEBGtest_utilization (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true roles = RoleSet { roles: {} } saga_id = dd4f75ce-0539-4224-8fc6-8244063d4551 saga_name = project-create saga_node = VpcCreateGateway
38512025-04-30T06:43:53.512ZDEBGtest_utilization (SagaExecutor): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true resource = Database result = Ok(()) saga_id = dd4f75ce-0539-4224-8fc6-8244063d4551 saga_name = project-create saga_node = VpcCreateGateway
38522025-04-30T06:43:53.515ZTRACtest_utilization (SagaExecutor): loading roles actor = Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. } actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet saga_id = dd4f75ce-0539-4224-8fc6-8244063d4551 saga_name = project-create saga_node = VpcCreateGateway
38532025-04-30T06:43:53.515ZTRACtest_utilization (SagaExecutor): authorize begin action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true resource = Database saga_id = dd4f75ce-0539-4224-8fc6-8244063d4551 saga_name = project-create saga_node = VpcCreateGateway
38542025-04-30T06:43:53.515ZDEBGtest_utilization (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true roles = RoleSet { roles: {} } saga_id = dd4f75ce-0539-4224-8fc6-8244063d4551 saga_name = project-create saga_node = VpcCreateGateway
38552025-04-30T06:43:53.515ZDEBGtest_utilization (SagaExecutor): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true resource = Database result = Ok(()) saga_id = dd4f75ce-0539-4224-8fc6-8244063d4551 saga_name = project-create saga_node = VpcCreateGateway
38562025-04-30T06:43:53.518ZDEBGtest_utilization (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin"), (Silo, 001de000-5110-4000-8000-000000000000, "admin")} } saga_id = dd4f75ce-0539-4224-8fc6-8244063d4551 saga_name = project-create saga_node = VpcCreateGateway
38572025-04-30T06:43:53.528ZDEBGtest_utilization (SagaExecutor): authorize result action = CreateChild actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true resource = InternetGateway { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000000, lookup_type: ById(001de000-5110-4000-8000-000000000000) }, key: 473b39f9-8669-45b3-b3d7-793f7e10fc31, lookup_type: ByName("utilization-test-project") }, key: cda41110-774f-439b-b99b-f715ebb74e33, lookup_type: ByName("default") }, key: 5567c57c-3b46-4ca0-968f-03632bf48df0, lookup_type: ById(5567c57c-3b46-4ca0-968f-03632bf48df0) } result = Ok(()) saga_id = dd4f75ce-0539-4224-8fc6-8244063d4551 saga_name = project-create saga_node = VpcCreateGateway
38582025-04-30T06:43:53.528ZTRACtest_utilization (SagaExecutor): authorize begin action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true resource = Database saga_id = dd4f75ce-0539-4224-8fc6-8244063d4551 saga_name = project-create saga_node = VpcCreateGateway
38592025-04-30T06:43:53.528ZDEBGtest_utilization (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true roles = RoleSet { roles: {} } saga_id = dd4f75ce-0539-4224-8fc6-8244063d4551 saga_name = project-create saga_node = VpcCreateGateway
38602025-04-30T06:43:53.529ZDEBGtest_utilization (SagaExecutor): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true resource = Database result = Ok(()) saga_id = dd4f75ce-0539-4224-8fc6-8244063d4551 saga_name = project-create saga_node = VpcCreateGateway
38612025-04-30T06:43:53.535ZDEBGtest_utilization (SEC): saga log event new_state = N017 succeeded sec_id = ac55d8d9-a298-4534-898b-beee1c9e4e52
38622025-04-30T06:43:53.535ZDEBGtest_utilization (SecStore): recording saga event event_type = Succeeded(Object {"key": String("5567c57c-3b46-4ca0-968f-03632bf48df0"), "lookup_type": Object {"ById": String("5567c57c-3b46-4ca0-968f-03632bf48df0")}, "parent": Object {"key": String("cda41110-774f-439b-b99b-f715ebb74e33"), "lookup_type": Object {"ByName": String("default")}, "parent": Object {"key": String("473b39f9-8669-45b3-b3d7-793f7e10fc31"), "lookup_type": Object {"ByName": String("utilization-test-project")}, "parent": Object {"key": String("001de000-5110-4000-8000-000000000000"), "lookup_type": Object {"ById": String("001de000-5110-4000-8000-000000000000")}, "parent": Null}}}}) node_id = 17 saga_id = dd4f75ce-0539-4224-8fc6-8244063d4551
38632025-04-30T06:43:53.549ZDEBGtest_utilization (SEC): saga log event new_state = N018 started sec_id = ac55d8d9-a298-4534-898b-beee1c9e4e52
38642025-04-30T06:43:53.549ZDEBGtest_utilization (SecStore): recording saga event event_type = Started node_id = 18 saga_id = dd4f75ce-0539-4224-8fc6-8244063d4551
38652025-04-30T06:43:53.556ZTRACtest_utilization (SagaExecutor): authorize begin action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true resource = Database saga_id = dd4f75ce-0539-4224-8fc6-8244063d4551 saga_name = project-create saga_node = VpcNotifySleds
38662025-04-30T06:43:53.556ZDEBGtest_utilization (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true roles = RoleSet { roles: {} } saga_id = dd4f75ce-0539-4224-8fc6-8244063d4551 saga_name = project-create saga_node = VpcNotifySleds
38672025-04-30T06:43:53.557ZDEBGtest_utilization (SagaExecutor): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true resource = Database result = Ok(()) saga_id = dd4f75ce-0539-4224-8fc6-8244063d4551 saga_name = project-create saga_node = VpcNotifySleds
38682025-04-30T06:43:53.562ZTRACtest_utilization (SagaExecutor): authorize begin action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true resource = Database saga_id = dd4f75ce-0539-4224-8fc6-8244063d4551 saga_name = project-create saga_node = VpcNotifySleds
38692025-04-30T06:43:53.562ZDEBGtest_utilization (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true roles = RoleSet { roles: {} } saga_id = dd4f75ce-0539-4224-8fc6-8244063d4551 saga_name = project-create saga_node = VpcNotifySleds
38702025-04-30T06:43:53.562ZDEBGtest_utilization (SagaExecutor): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true resource = Database result = Ok(()) saga_id = dd4f75ce-0539-4224-8fc6-8244063d4551 saga_name = project-create saga_node = VpcNotifySleds
38712025-04-30T06:43:53.568ZTRACtest_utilization (SagaExecutor): authorize begin action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true resource = Database saga_id = dd4f75ce-0539-4224-8fc6-8244063d4551 saga_name = project-create saga_node = VpcNotifySleds
38722025-04-30T06:43:53.569ZDEBGtest_utilization (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true roles = RoleSet { roles: {} } saga_id = dd4f75ce-0539-4224-8fc6-8244063d4551 saga_name = project-create saga_node = VpcNotifySleds
38732025-04-30T06:43:53.570ZDEBGtest_utilization (SagaExecutor): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true resource = Database result = Ok(()) saga_id = dd4f75ce-0539-4224-8fc6-8244063d4551 saga_name = project-create saga_node = VpcNotifySleds
38742025-04-30T06:43:53.590ZTRACtest_utilization (SagaExecutor): authorize begin action = ListChildren actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000000, lookup_type: ById(001de000-5110-4000-8000-000000000000) }, key: 473b39f9-8669-45b3-b3d7-793f7e10fc31, lookup_type: ById(473b39f9-8669-45b3-b3d7-793f7e10fc31) }, key: cda41110-774f-439b-b99b-f715ebb74e33, lookup_type: ByName("default") } saga_id = dd4f75ce-0539-4224-8fc6-8244063d4551 saga_name = project-create saga_node = VpcNotifySleds
38752025-04-30T06:43:53.590ZTRACtest_utilization (SagaExecutor): loading roles actor = Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. } actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true resource_id = 473b39f9-8669-45b3-b3d7-793f7e10fc31 resource_type = Project saga_id = dd4f75ce-0539-4224-8fc6-8244063d4551 saga_name = project-create saga_node = VpcNotifySleds
38762025-04-30T06:43:53.590ZTRACtest_utilization (SagaExecutor): authorize begin action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true resource = Database saga_id = dd4f75ce-0539-4224-8fc6-8244063d4551 saga_name = project-create saga_node = VpcNotifySleds
38772025-04-30T06:43:53.590ZDEBGtest_utilization (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true roles = RoleSet { roles: {} } saga_id = dd4f75ce-0539-4224-8fc6-8244063d4551 saga_name = project-create saga_node = VpcNotifySleds
38782025-04-30T06:43:53.591ZDEBGtest_utilization (SagaExecutor): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true resource = Database result = Ok(()) saga_id = dd4f75ce-0539-4224-8fc6-8244063d4551 saga_name = project-create saga_node = VpcNotifySleds
38792025-04-30T06:43:53.602ZTRACtest_utilization (SagaExecutor): loading roles actor = Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. } actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true resource_id = 001de000-5110-4000-8000-000000000000 resource_type = Silo saga_id = dd4f75ce-0539-4224-8fc6-8244063d4551 saga_name = project-create saga_node = VpcNotifySleds
38802025-04-30T06:43:53.602ZTRACtest_utilization (SagaExecutor): authorize begin action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true resource = Database saga_id = dd4f75ce-0539-4224-8fc6-8244063d4551 saga_name = project-create saga_node = VpcNotifySleds
38812025-04-30T06:43:53.602ZDEBGtest_utilization (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true roles = RoleSet { roles: {} } saga_id = dd4f75ce-0539-4224-8fc6-8244063d4551 saga_name = project-create saga_node = VpcNotifySleds
38822025-04-30T06:43:53.603ZDEBGtest_utilization (SagaExecutor): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true resource = Database result = Ok(()) saga_id = dd4f75ce-0539-4224-8fc6-8244063d4551 saga_name = project-create saga_node = VpcNotifySleds
38832025-04-30T06:43:53.608ZTRACtest_utilization (SagaExecutor): loading roles actor = Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. } actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet saga_id = dd4f75ce-0539-4224-8fc6-8244063d4551 saga_name = project-create saga_node = VpcNotifySleds
38842025-04-30T06:43:53.608ZTRACtest_utilization (SagaExecutor): authorize begin action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true resource = Database saga_id = dd4f75ce-0539-4224-8fc6-8244063d4551 saga_name = project-create saga_node = VpcNotifySleds
38852025-04-30T06:43:53.608ZDEBGtest_utilization (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true roles = RoleSet { roles: {} } saga_id = dd4f75ce-0539-4224-8fc6-8244063d4551 saga_name = project-create saga_node = VpcNotifySleds
38862025-04-30T06:43:53.610ZDEBGtest_utilization (SagaExecutor): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true resource = Database result = Ok(()) saga_id = dd4f75ce-0539-4224-8fc6-8244063d4551 saga_name = project-create saga_node = VpcNotifySleds
38872025-04-30T06:43:53.616ZDEBGtest_utilization (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin"), (Silo, 001de000-5110-4000-8000-000000000000, "admin")} } saga_id = dd4f75ce-0539-4224-8fc6-8244063d4551 saga_name = project-create saga_node = VpcNotifySleds
38882025-04-30T06:43:53.625ZDEBGtest_utilization (SagaExecutor): authorize result action = ListChildren actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000000, lookup_type: ById(001de000-5110-4000-8000-000000000000) }, key: 473b39f9-8669-45b3-b3d7-793f7e10fc31, lookup_type: ById(473b39f9-8669-45b3-b3d7-793f7e10fc31) }, key: cda41110-774f-439b-b99b-f715ebb74e33, lookup_type: ByName("default") } result = Ok(()) saga_id = dd4f75ce-0539-4224-8fc6-8244063d4551 saga_name = project-create saga_node = VpcNotifySleds
38892025-04-30T06:43:53.625ZTRACtest_utilization (SagaExecutor): authorize begin action = ListChildren actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000000, lookup_type: ById(001de000-5110-4000-8000-000000000000) }, key: 473b39f9-8669-45b3-b3d7-793f7e10fc31, lookup_type: ById(473b39f9-8669-45b3-b3d7-793f7e10fc31) }, key: cda41110-774f-439b-b99b-f715ebb74e33, lookup_type: ByName("default") } saga_id = dd4f75ce-0539-4224-8fc6-8244063d4551 saga_name = project-create saga_node = VpcNotifySleds
38902025-04-30T06:43:53.625ZTRACtest_utilization (SagaExecutor): loading roles actor = Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. } actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true resource_id = 473b39f9-8669-45b3-b3d7-793f7e10fc31 resource_type = Project saga_id = dd4f75ce-0539-4224-8fc6-8244063d4551 saga_name = project-create saga_node = VpcNotifySleds
38912025-04-30T06:43:53.625ZTRACtest_utilization (SagaExecutor): authorize begin action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true resource = Database saga_id = dd4f75ce-0539-4224-8fc6-8244063d4551 saga_name = project-create saga_node = VpcNotifySleds
38922025-04-30T06:43:53.625ZDEBGtest_utilization (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true roles = RoleSet { roles: {} } saga_id = dd4f75ce-0539-4224-8fc6-8244063d4551 saga_name = project-create saga_node = VpcNotifySleds
38932025-04-30T06:43:53.625ZDEBGtest_utilization (SagaExecutor): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true resource = Database result = Ok(()) saga_id = dd4f75ce-0539-4224-8fc6-8244063d4551 saga_name = project-create saga_node = VpcNotifySleds
38942025-04-30T06:43:53.629ZTRACtest_utilization (SagaExecutor): loading roles actor = Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. } actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true resource_id = 001de000-5110-4000-8000-000000000000 resource_type = Silo saga_id = dd4f75ce-0539-4224-8fc6-8244063d4551 saga_name = project-create saga_node = VpcNotifySleds
38952025-04-30T06:43:53.629ZTRACtest_utilization (SagaExecutor): authorize begin action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true resource = Database saga_id = dd4f75ce-0539-4224-8fc6-8244063d4551 saga_name = project-create saga_node = VpcNotifySleds
38962025-04-30T06:43:53.629ZDEBGtest_utilization (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true roles = RoleSet { roles: {} } saga_id = dd4f75ce-0539-4224-8fc6-8244063d4551 saga_name = project-create saga_node = VpcNotifySleds
38972025-04-30T06:43:53.629ZDEBGtest_utilization (SagaExecutor): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true resource = Database result = Ok(()) saga_id = dd4f75ce-0539-4224-8fc6-8244063d4551 saga_name = project-create saga_node = VpcNotifySleds
38982025-04-30T06:43:53.634ZTRACtest_utilization (SagaExecutor): loading roles actor = Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. } actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet saga_id = dd4f75ce-0539-4224-8fc6-8244063d4551 saga_name = project-create saga_node = VpcNotifySleds
38992025-04-30T06:43:53.634ZTRACtest_utilization (SagaExecutor): authorize begin action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true resource = Database saga_id = dd4f75ce-0539-4224-8fc6-8244063d4551 saga_name = project-create saga_node = VpcNotifySleds
39002025-04-30T06:43:53.634ZDEBGtest_utilization (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true roles = RoleSet { roles: {} } saga_id = dd4f75ce-0539-4224-8fc6-8244063d4551 saga_name = project-create saga_node = VpcNotifySleds
39012025-04-30T06:43:53.635ZDEBGtest_utilization (SagaExecutor): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true resource = Database result = Ok(()) saga_id = dd4f75ce-0539-4224-8fc6-8244063d4551 saga_name = project-create saga_node = VpcNotifySleds
39022025-04-30T06:43:53.640ZDEBGtest_utilization (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin"), (Silo, 001de000-5110-4000-8000-000000000000, "admin")} } saga_id = dd4f75ce-0539-4224-8fc6-8244063d4551 saga_name = project-create saga_node = VpcNotifySleds
39032025-04-30T06:43:53.655ZDEBGtest_utilization (SagaExecutor): authorize result action = ListChildren actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000000, lookup_type: ById(001de000-5110-4000-8000-000000000000) }, key: 473b39f9-8669-45b3-b3d7-793f7e10fc31, lookup_type: ById(473b39f9-8669-45b3-b3d7-793f7e10fc31) }, key: cda41110-774f-439b-b99b-f715ebb74e33, lookup_type: ByName("default") } result = Ok(()) saga_id = dd4f75ce-0539-4224-8fc6-8244063d4551 saga_name = project-create saga_node = VpcNotifySleds
39042025-04-30T06:43:53.655ZTRACtest_utilization (SagaExecutor): authorize begin action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true resource = Database saga_id = dd4f75ce-0539-4224-8fc6-8244063d4551 saga_name = project-create saga_node = VpcNotifySleds
39052025-04-30T06:43:53.655ZDEBGtest_utilization (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true roles = RoleSet { roles: {} } saga_id = dd4f75ce-0539-4224-8fc6-8244063d4551 saga_name = project-create saga_node = VpcNotifySleds
39062025-04-30T06:43:53.655ZDEBGtest_utilization (SagaExecutor): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true resource = Database result = Ok(()) saga_id = dd4f75ce-0539-4224-8fc6-8244063d4551 saga_name = project-create saga_node = VpcNotifySleds
39072025-04-30T06:43:53.663ZDEBGtest_utilization (nexus): resolved names for firewall rules instance_interfaces = {} subnet_interfaces = {} subnet_networks = {} vpc_interfaces = {}
39082025-04-30T06:43:53.663ZDEBGtest_utilization (nexus): resolved firewall rules for sled agents sled_agent_rules = []
39092025-04-30T06:43:53.663ZDEBGtest_utilization (nexus): resolved 0 rules for sleds
39102025-04-30T06:43:53.685ZDEBGtest_utilization (nexus): resolved sleds for vpc default vpc_to_sled = []
39112025-04-30T06:43:53.685ZDEBGtest_utilization (nexus): sending firewall rules to sled agents
39122025-04-30T06:43:53.685ZINFOtest_utilization (nexus): updated firewall rules on 0 sleds
39132025-04-30T06:43:53.685ZTRACtest_utilization (SagaExecutor): authorize begin action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true resource = Database saga_id = dd4f75ce-0539-4224-8fc6-8244063d4551 saga_name = project-create saga_node = VpcNotifySleds
39142025-04-30T06:43:53.685ZDEBGtest_utilization (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true roles = RoleSet { roles: {} } saga_id = dd4f75ce-0539-4224-8fc6-8244063d4551 saga_name = project-create saga_node = VpcNotifySleds
39152025-04-30T06:43:53.686ZDEBGtest_utilization (SagaExecutor): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true resource = Database result = Ok(()) saga_id = dd4f75ce-0539-4224-8fc6-8244063d4551 saga_name = project-create saga_node = VpcNotifySleds
39162025-04-30T06:43:53.690ZDEBGtest_utilization (SEC): saga log event new_state = N018 succeeded sec_id = ac55d8d9-a298-4534-898b-beee1c9e4e52
39172025-04-30T06:43:53.690ZDEBGtest_utilization (SecStore): recording saga event event_type = Succeeded(Null) node_id = 18 saga_id = dd4f75ce-0539-4224-8fc6-8244063d4551
39182025-04-30T06:43:53.704ZDEBGtest_utilization (SEC): saga log event new_state = N019 started sec_id = ac55d8d9-a298-4534-898b-beee1c9e4e52
39192025-04-30T06:43:53.704ZDEBGtest_utilization (SecStore): recording saga event event_type = Started node_id = 19 saga_id = dd4f75ce-0539-4224-8fc6-8244063d4551
39202025-04-30T06:43:53.707ZDEBGtest_utilization (SEC): saga log event new_state = N019 succeeded sec_id = ac55d8d9-a298-4534-898b-beee1c9e4e52
39212025-04-30T06:43:53.707ZDEBGtest_utilization (SecStore): recording saga event event_type = Succeeded(Null) node_id = 19 saga_id = dd4f75ce-0539-4224-8fc6-8244063d4551
39222025-04-30T06:43:53.711ZDEBGtest_utilization (SEC): saga log event new_state = N021 started sec_id = ac55d8d9-a298-4534-898b-beee1c9e4e52
39232025-04-30T06:43:53.711ZDEBGtest_utilization (SecStore): recording saga event event_type = Started node_id = 21 saga_id = dd4f75ce-0539-4224-8fc6-8244063d4551
39242025-04-30T06:43:53.714ZDEBGtest_utilization (SEC): saga log event new_state = N021 succeeded sec_id = ac55d8d9-a298-4534-898b-beee1c9e4e52
39252025-04-30T06:43:53.715ZDEBGtest_utilization (SecStore): recording saga event event_type = Succeeded(Null) node_id = 21 saga_id = dd4f75ce-0539-4224-8fc6-8244063d4551
39262025-04-30T06:43:53.717ZINFOtest_utilization (SEC): update for saga cached state new_state = Done saga_id = dd4f75ce-0539-4224-8fc6-8244063d4551 sec_id = ac55d8d9-a298-4534-898b-beee1c9e4e52
39272025-04-30T06:43:53.717ZINFOtest_utilization (SecStore): updating state new_state = done saga_id = dd4f75ce-0539-4224-8fc6-8244063d4551
39282025-04-30T06:43:53.726ZINFOtest_utilization (SEC): saga finished result = success saga_id = dd4f75ce-0539-4224-8fc6-8244063d4551 saga_name = project-create sec_id = ac55d8d9-a298-4534-898b-beee1c9e4e52
39292025-04-30T06:43:53.726ZINFOtest_utilization (SagaExecutor): saga finished saga_id = dd4f75ce-0539-4224-8fc6-8244063d4551 saga_name = project-create saga_result = SagaResult { saga_id: dd4f75ce-0539-4224-8fc6-8244063d4551, saga_log: SagaLog { saga_id: dd4f75ce-0539-4224-8fc6-8244063d4551, unwinding: false, events: [N020 started, N020 succeeded, N000 started, N000 succeeded, N001 started, N001 succeeded, N002 started, N002 succeeded, N003 started, N003 succeeded, N004 started, N004 succeeded, N005 started, N005 succeeded, N006 started, N006 succeeded, N007 started, N007 succeeded, N008 started, N008 succeeded, N009 started, N009 succeeded, N010 started, N010 succeeded, N011 started, N011 succeeded, N012 started, N012 succeeded, N013 started, N013 succeeded, N014 started, N014 succeeded, N015 started, N015 succeeded, N016 started, N016 succeeded, N017 started, N017 succeeded, N018 started, N018 succeeded, N019 started, N019 succeeded, N021 started, N021 succeeded], node_status: {0: Succeeded(Array [Object {"key": String("473b39f9-8669-45b3-b3d7-793f7e10fc31"), "lookup_type": Object {"ByName": String("utilization-test-project")}, "parent": Object {"key": String("001de000-5110-4000-8000-000000000000"), "lookup_type": Object {"ById": String("001de000-5110-4000-8000-000000000000")}, "parent": Null}}, Object {"identity": Object {"description": String("a pier"), "id": String("473b39f9-8669-45b3-b3d7-793f7e10fc31"), "name": String("utilization-test-project"), "time_created": String("2025-04-30T06:43:52.898759Z"), "time_deleted": Null, "time_modified": String("2025-04-30T06:43:52.898759Z")}, "rcgen": Number(1), "silo_id": String("001de000-5110-4000-8000-000000000000")}]), 1: Succeeded(Object {"authz_project": Object {"key": String("473b39f9-8669-45b3-b3d7-793f7e10fc31"), "lookup_type": Object {"ByName": String("utilization-test-project")}, "parent": Object {"key": String("001de000-5110-4000-8000-000000000000"), "lookup_type": Object {"ById": String("001de000-5110-4000-8000-000000000000")}, "parent": Null}}, "serialized_authn": Object {"kind": Object {"Authenticated": Array [Object {"actor": Object {"SiloUser": Object {"silo_id": String("001de000-5110-4000-8000-000000000000"), "silo_user_id": String("001de000-05e4-4000-8000-000000004007")}}}, Object {"mapped_fleet_roles": Object {}}]}}, "vpc_create": Object {"description": String("Default VPC"), "dns_name": String("default"), "ipv6_prefix": String("fdb8:6580:39ee::/48"), "name": String("default")}}), 2: Succeeded(Null), 3: Succeeded(String("cda41110-774f-439b-b99b-f715ebb74e33")), 4: Succeeded(String("fd90c8f9-ab47-4a3c-b34d-890ffe0ee4c4")), 5: Succeeded(String("e6961c2a-0f21-4758-9be0-412909d5abd1")), 6: Succeeded(String("a37eafa9-b3db-4363-aa0e-f96c1ac20163")), 7: Succeeded(String("e0639477-adf0-4eba-bf8b-9e814214f7fb")), 8: Succeeded(String("be0543da-af63-4252-b6b2-ce2bffd4dd49")), 9: Succeeded(String("5567c57c-3b46-4ca0-968f-03632bf48df0")), 10: Succeeded(Array [Object {"key": String("cda41110-774f-439b-b99b-f715ebb74e33"), "lookup_type": Object {"ByName": String("default")}, "parent": Object {"key": String("473b39f9-8669-45b3-b3d7-793f7e10fc31"), "lookup_type": Object {"ByName": String("utilization-test-project")}, "parent": Object {"key": String("001de000-5110-4000-8000-000000000000"), "lookup_type": Object {"ById": String("001de000-5110-4000-8000-000000000000")}, "parent": Null}}}, Object {"dns_name": String("default"), "firewall_gen": Number(1), "identity": Object {"description": String("Default VPC"), "id": String("cda41110-774f-439b-b99b-f715ebb74e33"), "name": String("default"), "time_created": String("2025-04-30T06:43:53.094070Z"), "time_deleted": Null, "time_modified": String("2025-04-30T06:43:53.094070Z")}, "ipv6_prefix": String("fdb8:6580:39ee::/48"), "project_id": String("473b39f9-8669-45b3-b3d7-793f7e10fc31"), "subnet_gen": Number(1), "system_router_id": String("fd90c8f9-ab47-4a3c-b34d-890ffe0ee4c4"), "vni": Number(11911937)}]), 11: Succeeded(Object {"key": String("fd90c8f9-ab47-4a3c-b34d-890ffe0ee4c4"), "lookup_type": Object {"ById": String("fd90c8f9-ab47-4a3c-b34d-890ffe0ee4c4")}, "parent": Object {"key": String("cda41110-774f-439b-b99b-f715ebb74e33"), "lookup_type": Object {"ByName": String("default")}, "parent": Object {"key": String("473b39f9-8669-45b3-b3d7-793f7e10fc31"), "lookup_type": Object {"ByName": String("utilization-test-project")}, "parent": Object {"key": String("001de000-5110-4000-8000-000000000000"), "lookup_type": Object {"ById": String("001de000-5110-4000-8000-000000000000")}, "parent": Null}}}}), 12: Succeeded(Null), 13: Succeeded(Null), 14: Succeeded(Array [Object {"key": String("be0543da-af63-4252-b6b2-ce2bffd4dd49"), "lookup_type": Object {"ById": String("be0543da-af63-4252-b6b2-ce2bffd4dd49")}, "parent": Object {"key": String("cda41110-774f-439b-b99b-f715ebb74e33"), "lookup_type": Object {"ByName": String("default")}, "parent": Object {"key": String("473b39f9-8669-45b3-b3d7-793f7e10fc31"), "lookup_type": Object {"ByName": String("utilization-test-project")}, "parent": Object {"key": String("001de000-5110-4000-8000-000000000000"), "lookup_type": Object {"ById": String("001de000-5110-4000-8000-000000000000")}, "parent": Null}}}}, Object {"custom_router_id": Null, "identity": Object {"description": String("The default subnet for default"), "id": String("be0543da-af63-4252-b6b2-ce2bffd4dd49"), "name": String("default"), "time_created": String("2025-04-30T06:43:53.314797Z"), "time_deleted": Null, "time_modified": String("2025-04-30T06:43:53.314797Z")}, "ipv4_block": String("172.30.0.0/22"), "ipv6_block": String("fdb8:6580:39ee::/64"), "rcgen": Number(1), "vpc_id": String("cda41110-774f-439b-b99b-f715ebb74e33")}]), 15: Succeeded(Object {"key": String("e0639477-adf0-4eba-bf8b-9e814214f7fb"), "lookup_type": Object {"ById": String("be0543da-af63-4252-b6b2-ce2bffd4dd49")}, "parent": Object {"key": String("fd90c8f9-ab47-4a3c-b34d-890ffe0ee4c4"), "lookup_type": Object {"ById": String("fd90c8f9-ab47-4a3c-b34d-890ffe0ee4c4")}, "parent": Object {"key": String("cda41110-774f-439b-b99b-f715ebb74e33"), "lookup_type": Object {"ByName": String("default")}, "parent": Object {"key": String("473b39f9-8669-45b3-b3d7-793f7e10fc31"), "lookup_type": Object {"ByName": String("utilization-test-project")}, "parent": Object {"key": String("001de000-5110-4000-8000-000000000000"), "lookup_type": Object {"ById": String("001de000-5110-4000-8000-000000000000")}, "parent": Null}}}}}), 16: Succeeded(Array [Object {"action": String("allow"), "direction": String("inbound"), "filter_hosts": Array [Object {"type": String("vpc"), "value": String("default")}], "filter_ports": Null, "filter_protocols": Null, "identity": Object {"description": String("allow inbound traffic to all instances within the VPC if originated within the VPC"), "id": String("168827d1-2d45-47a9-a589-52bf9bfc93aa"), "name": String("allow-internal-inbound"), "time_created": String("2025-04-30T06:43:53.401895291Z"), "time_deleted": Null, "time_modified": String("2025-04-30T06:43:53.401895291Z")}, "priority": Number(65534), "status": String("enabled"), "targets": Array [Object {"type": String("vpc"), "value": String("default")}], "vpc_id": String("cda41110-774f-439b-b99b-f715ebb74e33")}, Object {"action": String("allow"), "direction": String("inbound"), "filter_hosts": Null, "filter_ports": Array [String("22")], "filter_protocols": Array [String("TCP")], "identity": Object {"description": String("allow inbound TCP connections on port 22 from anywhere"), "id": String("cab9a6af-a8a8-4fa2-80b1-0e0b67193155"), "name": String("allow-ssh"), "time_created": String("2025-04-30T06:43:53.401911463Z"), "time_deleted": Null, "time_modified": String("2025-04-30T06:43:53.401911463Z")}, "priority": Number(65534), "status": String("enabled"), "targets": Array [Object {"type": String("vpc"), "value": String("default")}], "vpc_id": String("cda41110-774f-439b-b99b-f715ebb74e33")}, Object {"action": String("allow"), "direction": String("inbound"), "filter_hosts": Null, "filter_ports": Null, "filter_protocols": Array [String("ICMP")], "identity": Object {"description": String("allow inbound ICMP traffic from anywhere"), "id": String("540073f5-235b-41db-9c1a-2d426500061e"), "name": String("allow-icmp"), "time_created": String("2025-04-30T06:43:53.401918397Z"), "time_deleted": Null, "time_modified": String("2025-04-30T06:43:53.401918397Z")}, "priority": Number(65534), "status": String("enabled"), "targets": Array [Object {"type": String("vpc"), "value": String("default")}], "vpc_id": String("cda41110-774f-439b-b99b-f715ebb74e33")}]), 17: Succeeded(Object {"key": String("5567c57c-3b46-4ca0-968f-03632bf48df0"), "lookup_type": Object {"ById": String("5567c57c-3b46-4ca0-968f-03632bf48df0")}, "parent": Object {"key": String("cda41110-774f-439b-b99b-f715ebb74e33"), "lookup_type": Object {"ByName": String("default")}, "parent": Object {"key": String("473b39f9-8669-45b3-b3d7-793f7e10fc31"), "lookup_type": Object {"ByName": String("utilization-test-project")}, "parent": Object {"key": String("001de000-5110-4000-8000-000000000000"), "lookup_type": Object {"ById": String("001de000-5110-4000-8000-000000000000")}, "parent": Null}}}}), 18: Succeeded(Null), 19: Succeeded(Null), 20: Succeeded(Null), 21: Succeeded(Null)} }, kind: Ok(SagaResultOk { saga_output: Null, node_outputs: {"default_internet_gateway_id": String("5567c57c-3b46-4ca0-968f-03632bf48df0"), "default_subnet_id": String("be0543da-af63-4252-b6b2-ce2bffd4dd49"), "default_v4_route_id": String("e6961c2a-0f21-4758-9be0-412909d5abd1"), "default_v6_route_id": String("a37eafa9-b3db-4363-aa0e-f96c1ac20163"), "firewall": Array [Object {"action": String("allow"), "direction": String("inbound"), "filter_hosts": Array [Object {"type": String("vpc"), "value": String("default")}], "filter_ports": Null, "filter_protocols": Null, "identity": Object {"description": String("allow inbound traffic to all instances within the VPC if originated within the VPC"), "id": String("168827d1-2d45-47a9-a589-52bf9bfc93aa"), "name": String("allow-internal-inbound"), "time_created": String("2025-04-30T06:43:53.401895291Z"), "time_deleted": Null, "time_modified": String("2025-04-30T06:43:53.401895291Z")}, "priority": Number(65534), "status": String("enabled"), "targets": Array [Object {"type": String("vpc"), "value": String("default")}], "vpc_id": String("cda41110-774f-439b-b99b-f715ebb74e33")}, Object {"action": String("allow"), "direction": String("inbound"), "filter_hosts": Null, "filter_ports": Array [String("22")], "filter_protocols": Array [String("TCP")], "identity": Object {"description": String("allow inbound TCP connections on port 22 from anywhere"), "id": String("cab9a6af-a8a8-4fa2-80b1-0e0b67193155"), "name": String("allow-ssh"), "time_created": String("2025-04-30T06:43:53.401911463Z"), "time_deleted": Null, "time_modified": String("2025-04-30T06:43:53.401911463Z")}, "priority": Number(65534), "status": String("enabled"), "targets": Array [Object {"type": String("vpc"), "value": String("default")}], "vpc_id": String("cda41110-774f-439b-b99b-f715ebb74e33")}, Object {"action": String("allow"), "direction": String("inbound"), "filter_hosts": Null, "filter_ports": Null, "filter_protocols": Array [String("ICMP")], "identity": Object {"description": String("allow inbound ICMP traffic from anywhere"), "id": String("540073f5-235b-41db-9c1a-2d426500061e"), "name": String("allow-icmp"), "time_created": String("2025-04-30T06:43:53.401918397Z"), "time_deleted": Null, "time_modified": String("2025-04-30T06:43:53.401918397Z")}, "priority": Number(65534), "status": String("enabled"), "targets": Array [Object {"type": String("vpc"), "value": String("default")}], "vpc_id": String("cda41110-774f-439b-b99b-f715ebb74e33")}], "gateway": Object {"key": String("5567c57c-3b46-4ca0-968f-03632bf48df0"), "lookup_type": Object {"ById": String("5567c57c-3b46-4ca0-968f-03632bf48df0")}, "parent": Object {"key": String("cda41110-774f-439b-b99b-f715ebb74e33"), "lookup_type": Object {"ByName": String("default")}, "parent": Object {"key": String("473b39f9-8669-45b3-b3d7-793f7e10fc31"), "lookup_type": Object {"ByName": String("utilization-test-project")}, "parent": Object {"key": String("001de000-5110-4000-8000-000000000000"), "lookup_type": Object {"ById": String("001de000-5110-4000-8000-000000000000")}, "parent": Null}}}}, "no_result": Null, "project": Array [Object {"key": String("473b39f9-8669-45b3-b3d7-793f7e10fc31"), "lookup_type": Object {"ByName": String("utilization-test-project")}, "parent": Object {"key": String("001de000-5110-4000-8000-000000000000"), "lookup_type": Object {"ById": String("001de000-5110-4000-8000-000000000000")}, "parent": Null}}, Object {"identity": Object {"description": String("a pier"), "id": String("473b39f9-8669-45b3-b3d7-793f7e10fc31"), "name": String("utilization-test-project"), "time_created": String("2025-04-30T06:43:52.898759Z"), "time_deleted": Null, "time_modified": String("2025-04-30T06:43:52.898759Z")}, "rcgen": Number(1), "silo_id": String("001de000-5110-4000-8000-000000000000")}], "route": Object {"key": String("e0639477-adf0-4eba-bf8b-9e814214f7fb"), "lookup_type": Object {"ById": String("be0543da-af63-4252-b6b2-ce2bffd4dd49")}, "parent": Object {"key": String("fd90c8f9-ab47-4a3c-b34d-890ffe0ee4c4"), "lookup_type": Object {"ById": String("fd90c8f9-ab47-4a3c-b34d-890ffe0ee4c4")}, "parent": Object {"key": String("cda41110-774f-439b-b99b-f715ebb74e33"), "lookup_type": Object {"ByName": String("default")}, "parent": Object {"key": String("473b39f9-8669-45b3-b3d7-793f7e10fc31"), "lookup_type": Object {"ByName": String("utilization-test-project")}, "parent": Object {"key": String("001de000-5110-4000-8000-000000000000"), "lookup_type": Object {"ById": String("001de000-5110-4000-8000-000000000000")}, "parent": Null}}}}}, "route4": Null, "route6": Null, "router": Object {"key": String("fd90c8f9-ab47-4a3c-b34d-890ffe0ee4c4"), "lookup_type": Object {"ById": String("fd90c8f9-ab47-4a3c-b34d-890ffe0ee4c4")}, "parent": Object {"key": String("cda41110-774f-439b-b99b-f715ebb74e33"), "lookup_type": Object {"ByName": String("default")}, "parent": Object {"key": String("473b39f9-8669-45b3-b3d7-793f7e10fc31"), "lookup_type": Object {"ByName": String("utilization-test-project")}, "parent": Object {"key": String("001de000-5110-4000-8000-000000000000"), "lookup_type": Object {"ById": String("001de000-5110-4000-8000-000000000000")}, "parent": Null}}}}, "subnet": Array [Object {"key": String("be0543da-af63-4252-b6b2-ce2bffd4dd49"), "lookup_type": Object {"ById": String("be0543da-af63-4252-b6b2-ce2bffd4dd49")}, "parent": Object {"key": String("cda41110-774f-439b-b99b-f715ebb74e33"), "lookup_type": Object {"ByName": String("default")}, "parent": Object {"key": String("473b39f9-8669-45b3-b3d7-793f7e10fc31"), "lookup_type": Object {"ByName": String("utilization-test-project")}, "parent": Object {"key": String("001de000-5110-4000-8000-000000000000"), "lookup_type": Object {"ById": String("001de000-5110-4000-8000-000000000000")}, "parent": Null}}}}, Object {"custom_router_id": Null, "identity": Object {"description": String("The default subnet for default"), "id": String("be0543da-af63-4252-b6b2-ce2bffd4dd49"), "name": String("default"), "time_created": String("2025-04-30T06:43:53.314797Z"), "time_deleted": Null, "time_modified": String("2025-04-30T06:43:53.314797Z")}, "ipv4_block": String("172.30.0.0/22"), "ipv6_block": String("fdb8:6580:39ee::/64"), "rcgen": Number(1), "vpc_id": String("cda41110-774f-439b-b99b-f715ebb74e33")}], "subnet_route_id": String("e0639477-adf0-4eba-bf8b-9e814214f7fb"), "system_router_id": String("fd90c8f9-ab47-4a3c-b34d-890ffe0ee4c4"), "vpc": Null, "vpc_create_params": Object {"authz_project": Object {"key": String("473b39f9-8669-45b3-b3d7-793f7e10fc31"), "lookup_type": Object {"ByName": String("utilization-test-project")}, "parent": Object {"key": String("001de000-5110-4000-8000-000000000000"), "lookup_type": Object {"ById": String("001de000-5110-4000-8000-000000000000")}, "parent": Null}}, "serialized_authn": Object {"kind": Object {"Authenticated": Array [Object {"actor": Object {"SiloUser": Object {"silo_id": String("001de000-5110-4000-8000-000000000000"), "silo_user_id": String("001de000-05e4-4000-8000-000000004007")}}}, Object {"mapped_fleet_roles": Object {}}]}}, "vpc_create": Object {"description": String("Default VPC"), "dns_name": String("default"), "ipv6_prefix": String("fdb8:6580:39ee::/48"), "name": String("default")}}, "vpc_id": String("cda41110-774f-439b-b99b-f715ebb74e33")} }) }
39302025-04-30T06:43:53.728ZINFOtest_utilization (dropshot_external): request completed latency_us = 1023515 local_addr = 127.0.0.1:46348 method = POST remote_addr = 127.0.0.1:41283 req_id = 07ae243d-4bd1-44d6-8cea-cee37952d5a7 response_code = 201 uri = /v1/projects
39312025-04-30T06:43:53.728ZINFOtest_utilization (external client test context): client received response status = 201
39322025-04-30T06:43:53.730ZINFOtest_utilization (external client test context): client request body = Body { inner: BoxBody } method = POST uri = http://127.0.0.1:46348/v1/instances?project=utilization-test-project
39332025-04-30T06:43:53.730ZTRACtest_utilization (dropshot_external): incoming request local_addr = 127.0.0.1:46348 method = POST remote_addr = 127.0.0.1:41283 req_id = bf272e5e-cbe5-41d9-b505-34877ffe12d2 uri = /v1/instances?project=utilization-test-project
39342025-04-30T06:43:53.731ZTRACtest_utilization (dropshot_external): authn: trying SchemeName("spoof") local_addr = 127.0.0.1:46348 method = POST remote_addr = 127.0.0.1:41283 req_id = bf272e5e-cbe5-41d9-b505-34877ffe12d2 uri = /v1/instances?project=utilization-test-project
39352025-04-30T06:43:53.731ZDEBGtest_utilization (dropshot_external): looking up silo for user local_addr = 127.0.0.1:46348 method = POST remote_addr = 127.0.0.1:41283 req_id = bf272e5e-cbe5-41d9-b505-34877ffe12d2 silo_user_id = 001de000-05e4-4000-8000-000000004007 uri = /v1/instances?project=utilization-test-project
39362025-04-30T06:43:53.731ZTRACtest_utilization (ExternalAuthn): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database
39372025-04-30T06:43:53.732ZDEBGtest_utilization (ExternalAuthn): roles roles = RoleSet { roles: {} }
39382025-04-30T06:43:53.733ZDEBGtest_utilization (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
39392025-04-30T06:43:53.736ZTRACtest_utilization (ExternalAuthn): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database
39402025-04-30T06:43:53.736ZDEBGtest_utilization (ExternalAuthn): roles roles = RoleSet { roles: {} }
39412025-04-30T06:43:53.737ZDEBGtest_utilization (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
39422025-04-30T06:43:53.740ZTRACtest_utilization (ExternalAuthn): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = SiloUser { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000000, lookup_type: ById(001de000-5110-4000-8000-000000000000) }, key: 001de000-05e4-4000-8000-000000004007, lookup_type: ById(001de000-05e4-4000-8000-000000004007) }
39432025-04-30T06:43:53.740ZTRACtest_utilization (ExternalAuthn): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. } resource_id = 001de000-5110-4000-8000-000000000000 resource_type = Silo
39442025-04-30T06:43:53.740ZTRACtest_utilization (ExternalAuthn): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database
39452025-04-30T06:43:53.740ZDEBGtest_utilization (ExternalAuthn): roles roles = RoleSet { roles: {} }
39462025-04-30T06:43:53.742ZDEBGtest_utilization (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
39472025-04-30T06:43:53.745ZTRACtest_utilization (ExternalAuthn): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
39482025-04-30T06:43:53.745ZTRACtest_utilization (ExternalAuthn): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database
39492025-04-30T06:43:53.745ZDEBGtest_utilization (ExternalAuthn): roles roles = RoleSet { roles: {} }
39502025-04-30T06:43:53.747ZDEBGtest_utilization (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
39512025-04-30T06:43:53.750ZDEBGtest_utilization (ExternalAuthn): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "external-authenticator")} }
39522025-04-30T06:43:53.755ZDEBGtest_utilization (ExternalAuthn): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = SiloUser { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000000, lookup_type: ById(001de000-5110-4000-8000-000000000000) }, key: 001de000-05e4-4000-8000-000000004007, lookup_type: ById(001de000-05e4-4000-8000-000000004007) } result = Ok(())
39532025-04-30T06:43:53.755ZTRACtest_utilization (ExternalAuthn): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database
39542025-04-30T06:43:53.755ZDEBGtest_utilization (ExternalAuthn): roles roles = RoleSet { roles: {} }
39552025-04-30T06:43:53.755ZDEBGtest_utilization (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
39562025-04-30T06:43:53.758ZTRACtest_utilization (ExternalAuthn): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000000, lookup_type: ById(001de000-5110-4000-8000-000000000000) }
39572025-04-30T06:43:53.758ZTRACtest_utilization (ExternalAuthn): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. } resource_id = 001de000-5110-4000-8000-000000000000 resource_type = Silo
39582025-04-30T06:43:53.758ZTRACtest_utilization (ExternalAuthn): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database
39592025-04-30T06:43:53.758ZDEBGtest_utilization (ExternalAuthn): roles roles = RoleSet { roles: {} }
39602025-04-30T06:43:53.759ZDEBGtest_utilization (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
39612025-04-30T06:43:53.762ZTRACtest_utilization (ExternalAuthn): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
39622025-04-30T06:43:53.762ZTRACtest_utilization (ExternalAuthn): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database
39632025-04-30T06:43:53.762ZDEBGtest_utilization (ExternalAuthn): roles roles = RoleSet { roles: {} }
39642025-04-30T06:43:53.763ZDEBGtest_utilization (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
39652025-04-30T06:43:53.765ZDEBGtest_utilization (ExternalAuthn): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "external-authenticator")} }
39662025-04-30T06:43:53.773ZDEBGtest_utilization (ExternalAuthn): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000000, lookup_type: ById(001de000-5110-4000-8000-000000000000) } result = Ok(())
39672025-04-30T06:43:53.773ZTRACtest_utilization (dropshot_external): authn result: Ok(Context { kind: Authenticated(Details { actor: Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. } }, Some(SiloAuthnPolicy { mapped_fleet_roles: {} })), schemes_tried: [SchemeName("spoof")] }) local_addr = 127.0.0.1:46348 method = POST remote_addr = 127.0.0.1:41283 req_id = bf272e5e-cbe5-41d9-b505-34877ffe12d2 uri = /v1/instances?project=utilization-test-project
39682025-04-30T06:43:53.773ZTRACtest_utilization (dropshot_external): authorize begin action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true local_addr = 127.0.0.1:46348 method = POST remote_addr = 127.0.0.1:41283 req_id = bf272e5e-cbe5-41d9-b505-34877ffe12d2 resource = Database uri = /v1/instances?project=utilization-test-project
39692025-04-30T06:43:53.773ZDEBGtest_utilization (dropshot_external): roles actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true local_addr = 127.0.0.1:46348 method = POST remote_addr = 127.0.0.1:41283 req_id = bf272e5e-cbe5-41d9-b505-34877ffe12d2 roles = RoleSet { roles: {} } uri = /v1/instances?project=utilization-test-project
39702025-04-30T06:43:53.774ZDEBGtest_utilization (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true local_addr = 127.0.0.1:46348 method = POST remote_addr = 127.0.0.1:41283 req_id = bf272e5e-cbe5-41d9-b505-34877ffe12d2 resource = Database result = Ok(()) uri = /v1/instances?project=utilization-test-project
39712025-04-30T06:43:53.779ZTRACtest_utilization (dropshot_external): authorize begin action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true local_addr = 127.0.0.1:46348 method = POST remote_addr = 127.0.0.1:41283 req_id = bf272e5e-cbe5-41d9-b505-34877ffe12d2 resource = Database uri = /v1/instances?project=utilization-test-project
39722025-04-30T06:43:53.779ZDEBGtest_utilization (dropshot_external): roles actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true local_addr = 127.0.0.1:46348 method = POST remote_addr = 127.0.0.1:41283 req_id = bf272e5e-cbe5-41d9-b505-34877ffe12d2 roles = RoleSet { roles: {} } uri = /v1/instances?project=utilization-test-project
39732025-04-30T06:43:53.784ZDEBGtest_utilization (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true local_addr = 127.0.0.1:46348 method = POST remote_addr = 127.0.0.1:41283 req_id = bf272e5e-cbe5-41d9-b505-34877ffe12d2 resource = Database result = Ok(()) uri = /v1/instances?project=utilization-test-project
39742025-04-30T06:43:53.791ZTRACtest_utilization (dropshot_external): authorize begin action = CreateChild actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true local_addr = 127.0.0.1:46348 method = POST remote_addr = 127.0.0.1:41283 req_id = bf272e5e-cbe5-41d9-b505-34877ffe12d2 resource = Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000000, lookup_type: ById(001de000-5110-4000-8000-000000000000) }, key: 473b39f9-8669-45b3-b3d7-793f7e10fc31, lookup_type: ByName("utilization-test-project") } uri = /v1/instances?project=utilization-test-project
39752025-04-30T06:43:53.791ZTRACtest_utilization (dropshot_external): loading roles actor = Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. } actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true local_addr = 127.0.0.1:46348 method = POST remote_addr = 127.0.0.1:41283 req_id = bf272e5e-cbe5-41d9-b505-34877ffe12d2 resource_id = 473b39f9-8669-45b3-b3d7-793f7e10fc31 resource_type = Project uri = /v1/instances?project=utilization-test-project
39762025-04-30T06:43:53.791ZTRACtest_utilization (dropshot_external): authorize begin action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true local_addr = 127.0.0.1:46348 method = POST remote_addr = 127.0.0.1:41283 req_id = bf272e5e-cbe5-41d9-b505-34877ffe12d2 resource = Database uri = /v1/instances?project=utilization-test-project
39772025-04-30T06:43:53.792ZDEBGtest_utilization (dropshot_external): roles actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true local_addr = 127.0.0.1:46348 method = POST remote_addr = 127.0.0.1:41283 req_id = bf272e5e-cbe5-41d9-b505-34877ffe12d2 roles = RoleSet { roles: {} } uri = /v1/instances?project=utilization-test-project
39782025-04-30T06:43:53.793ZDEBGtest_utilization (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true local_addr = 127.0.0.1:46348 method = POST remote_addr = 127.0.0.1:41283 req_id = bf272e5e-cbe5-41d9-b505-34877ffe12d2 resource = Database result = Ok(()) uri = /v1/instances?project=utilization-test-project
39792025-04-30T06:43:53.808ZTRACtest_utilization (dropshot_external): loading roles actor = Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. } actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true local_addr = 127.0.0.1:46348 method = POST remote_addr = 127.0.0.1:41283 req_id = bf272e5e-cbe5-41d9-b505-34877ffe12d2 resource_id = 001de000-5110-4000-8000-000000000000 resource_type = Silo uri = /v1/instances?project=utilization-test-project
39802025-04-30T06:43:53.809ZTRACtest_utilization (dropshot_external): authorize begin action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true local_addr = 127.0.0.1:46348 method = POST remote_addr = 127.0.0.1:41283 req_id = bf272e5e-cbe5-41d9-b505-34877ffe12d2 resource = Database uri = /v1/instances?project=utilization-test-project
39812025-04-30T06:43:53.809ZDEBGtest_utilization (dropshot_external): roles actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true local_addr = 127.0.0.1:46348 method = POST remote_addr = 127.0.0.1:41283 req_id = bf272e5e-cbe5-41d9-b505-34877ffe12d2 roles = RoleSet { roles: {} } uri = /v1/instances?project=utilization-test-project
39822025-04-30T06:43:53.809ZDEBGtest_utilization (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true local_addr = 127.0.0.1:46348 method = POST remote_addr = 127.0.0.1:41283 req_id = bf272e5e-cbe5-41d9-b505-34877ffe12d2 resource = Database result = Ok(()) uri = /v1/instances?project=utilization-test-project
39832025-04-30T06:43:53.814ZTRACtest_utilization (dropshot_external): loading roles actor = Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. } actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true local_addr = 127.0.0.1:46348 method = POST remote_addr = 127.0.0.1:41283 req_id = bf272e5e-cbe5-41d9-b505-34877ffe12d2 resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet uri = /v1/instances?project=utilization-test-project
39842025-04-30T06:43:53.814ZTRACtest_utilization (dropshot_external): authorize begin action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true local_addr = 127.0.0.1:46348 method = POST remote_addr = 127.0.0.1:41283 req_id = bf272e5e-cbe5-41d9-b505-34877ffe12d2 resource = Database uri = /v1/instances?project=utilization-test-project
39852025-04-30T06:43:53.814ZDEBGtest_utilization (dropshot_external): roles actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true local_addr = 127.0.0.1:46348 method = POST remote_addr = 127.0.0.1:41283 req_id = bf272e5e-cbe5-41d9-b505-34877ffe12d2 roles = RoleSet { roles: {} } uri = /v1/instances?project=utilization-test-project
39862025-04-30T06:43:53.815ZDEBGtest_utilization (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true local_addr = 127.0.0.1:46348 method = POST remote_addr = 127.0.0.1:41283 req_id = bf272e5e-cbe5-41d9-b505-34877ffe12d2 resource = Database result = Ok(()) uri = /v1/instances?project=utilization-test-project
39872025-04-30T06:43:53.819ZDEBGtest_utilization (dropshot_external): roles actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true local_addr = 127.0.0.1:46348 method = POST remote_addr = 127.0.0.1:41283 req_id = bf272e5e-cbe5-41d9-b505-34877ffe12d2 roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin"), (Silo, 001de000-5110-4000-8000-000000000000, "admin")} } uri = /v1/instances?project=utilization-test-project
39882025-04-30T06:43:53.826ZDEBGtest_utilization (dropshot_external): authorize result action = CreateChild actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true local_addr = 127.0.0.1:46348 method = POST remote_addr = 127.0.0.1:41283 req_id = bf272e5e-cbe5-41d9-b505-34877ffe12d2 resource = Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000000, lookup_type: ById(001de000-5110-4000-8000-000000000000) }, key: 473b39f9-8669-45b3-b3d7-793f7e10fc31, lookup_type: ByName("utilization-test-project") } result = Ok(()) uri = /v1/instances?project=utilization-test-project
39892025-04-30T06:43:53.826ZTRACtest_utilization (dropshot_external): authorize begin action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true local_addr = 127.0.0.1:46348 method = POST remote_addr = 127.0.0.1:41283 req_id = bf272e5e-cbe5-41d9-b505-34877ffe12d2 resource = Database uri = /v1/instances?project=utilization-test-project
39902025-04-30T06:43:53.826ZDEBGtest_utilization (dropshot_external): roles actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true local_addr = 127.0.0.1:46348 method = POST remote_addr = 127.0.0.1:41283 req_id = bf272e5e-cbe5-41d9-b505-34877ffe12d2 roles = RoleSet { roles: {} } uri = /v1/instances?project=utilization-test-project
39912025-04-30T06:43:53.826ZDEBGtest_utilization (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true local_addr = 127.0.0.1:46348 method = POST remote_addr = 127.0.0.1:41283 req_id = bf272e5e-cbe5-41d9-b505-34877ffe12d2 resource = Database result = Ok(()) uri = /v1/instances?project=utilization-test-project
39922025-04-30T06:43:53.829ZTRACtest_utilization (dropshot_external): authorize begin action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true local_addr = 127.0.0.1:46348 method = POST remote_addr = 127.0.0.1:41283 req_id = bf272e5e-cbe5-41d9-b505-34877ffe12d2 resource = Database uri = /v1/instances?project=utilization-test-project
39932025-04-30T06:43:53.829ZDEBGtest_utilization (dropshot_external): roles actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true local_addr = 127.0.0.1:46348 method = POST remote_addr = 127.0.0.1:41283 req_id = bf272e5e-cbe5-41d9-b505-34877ffe12d2 roles = RoleSet { roles: {} } uri = /v1/instances?project=utilization-test-project
39942025-04-30T06:43:53.831ZDEBGtest_utilization (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true local_addr = 127.0.0.1:46348 method = POST remote_addr = 127.0.0.1:41283 req_id = bf272e5e-cbe5-41d9-b505-34877ffe12d2 resource = Database result = Ok(()) uri = /v1/instances?project=utilization-test-project
39952025-04-30T06:43:53.835ZTRACtest_utilization (dropshot_external): authorize begin action = ListChildren actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true local_addr = 127.0.0.1:46348 method = POST remote_addr = 127.0.0.1:41283 req_id = bf272e5e-cbe5-41d9-b505-34877ffe12d2 resource = SiloUser { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000000, lookup_type: ById(001de000-5110-4000-8000-000000000000) }, key: 001de000-05e4-4000-8000-000000004007, lookup_type: ById(001de000-05e4-4000-8000-000000004007) } uri = /v1/instances?project=utilization-test-project
39962025-04-30T06:43:53.835ZTRACtest_utilization (dropshot_external): loading roles actor = Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. } actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true local_addr = 127.0.0.1:46348 method = POST remote_addr = 127.0.0.1:41283 req_id = bf272e5e-cbe5-41d9-b505-34877ffe12d2 resource_id = 001de000-5110-4000-8000-000000000000 resource_type = Silo uri = /v1/instances?project=utilization-test-project
39972025-04-30T06:43:53.835ZTRACtest_utilization (dropshot_external): authorize begin action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true local_addr = 127.0.0.1:46348 method = POST remote_addr = 127.0.0.1:41283 req_id = bf272e5e-cbe5-41d9-b505-34877ffe12d2 resource = Database uri = /v1/instances?project=utilization-test-project
39982025-04-30T06:43:53.835ZDEBGtest_utilization (dropshot_external): roles actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true local_addr = 127.0.0.1:46348 method = POST remote_addr = 127.0.0.1:41283 req_id = bf272e5e-cbe5-41d9-b505-34877ffe12d2 roles = RoleSet { roles: {} } uri = /v1/instances?project=utilization-test-project
39992025-04-30T06:43:53.836ZDEBGtest_utilization (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true local_addr = 127.0.0.1:46348 method = POST remote_addr = 127.0.0.1:41283 req_id = bf272e5e-cbe5-41d9-b505-34877ffe12d2 resource = Database result = Ok(()) uri = /v1/instances?project=utilization-test-project
40002025-04-30T06:43:53.839ZTRACtest_utilization (dropshot_external): loading roles actor = Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. } actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true local_addr = 127.0.0.1:46348 method = POST remote_addr = 127.0.0.1:41283 req_id = bf272e5e-cbe5-41d9-b505-34877ffe12d2 resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet uri = /v1/instances?project=utilization-test-project
40012025-04-30T06:43:53.839ZTRACtest_utilization (dropshot_external): authorize begin action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true local_addr = 127.0.0.1:46348 method = POST remote_addr = 127.0.0.1:41283 req_id = bf272e5e-cbe5-41d9-b505-34877ffe12d2 resource = Database uri = /v1/instances?project=utilization-test-project
40022025-04-30T06:43:53.839ZDEBGtest_utilization (dropshot_external): roles actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true local_addr = 127.0.0.1:46348 method = POST remote_addr = 127.0.0.1:41283 req_id = bf272e5e-cbe5-41d9-b505-34877ffe12d2 roles = RoleSet { roles: {} } uri = /v1/instances?project=utilization-test-project
40032025-04-30T06:43:53.840ZDEBGtest_utilization (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true local_addr = 127.0.0.1:46348 method = POST remote_addr = 127.0.0.1:41283 req_id = bf272e5e-cbe5-41d9-b505-34877ffe12d2 resource = Database result = Ok(()) uri = /v1/instances?project=utilization-test-project
40042025-04-30T06:43:53.844ZDEBGtest_utilization (dropshot_external): roles actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true local_addr = 127.0.0.1:46348 method = POST remote_addr = 127.0.0.1:41283 req_id = bf272e5e-cbe5-41d9-b505-34877ffe12d2 roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin"), (Silo, 001de000-5110-4000-8000-000000000000, "admin")} } uri = /v1/instances?project=utilization-test-project
40052025-04-30T06:43:53.847ZDEBGtest_utilization (dropshot_external): authorize result action = ListChildren actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true local_addr = 127.0.0.1:46348 method = POST remote_addr = 127.0.0.1:41283 req_id = bf272e5e-cbe5-41d9-b505-34877ffe12d2 resource = SiloUser { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000000, lookup_type: ById(001de000-5110-4000-8000-000000000000) }, key: 001de000-05e4-4000-8000-000000004007, lookup_type: ById(001de000-05e4-4000-8000-000000004007) } result = Ok(()) uri = /v1/instances?project=utilization-test-project
40062025-04-30T06:43:53.847ZTRACtest_utilization (dropshot_external): authorize begin action = ListChildren actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true local_addr = 127.0.0.1:46348 method = POST remote_addr = 127.0.0.1:41283 req_id = bf272e5e-cbe5-41d9-b505-34877ffe12d2 resource = SiloUser { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000000, lookup_type: ById(001de000-5110-4000-8000-000000000000) }, key: 001de000-05e4-4000-8000-000000004007, lookup_type: ById(001de000-05e4-4000-8000-000000004007) } uri = /v1/instances?project=utilization-test-project
40072025-04-30T06:43:53.847ZTRACtest_utilization (dropshot_external): loading roles actor = Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. } actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true local_addr = 127.0.0.1:46348 method = POST remote_addr = 127.0.0.1:41283 req_id = bf272e5e-cbe5-41d9-b505-34877ffe12d2 resource_id = 001de000-5110-4000-8000-000000000000 resource_type = Silo uri = /v1/instances?project=utilization-test-project
40082025-04-30T06:43:53.847ZTRACtest_utilization (dropshot_external): authorize begin action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true local_addr = 127.0.0.1:46348 method = POST remote_addr = 127.0.0.1:41283 req_id = bf272e5e-cbe5-41d9-b505-34877ffe12d2 resource = Database uri = /v1/instances?project=utilization-test-project
40092025-04-30T06:43:53.847ZDEBGtest_utilization (dropshot_external): roles actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true local_addr = 127.0.0.1:46348 method = POST remote_addr = 127.0.0.1:41283 req_id = bf272e5e-cbe5-41d9-b505-34877ffe12d2 roles = RoleSet { roles: {} } uri = /v1/instances?project=utilization-test-project
40102025-04-30T06:43:53.847ZDEBGtest_utilization (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true local_addr = 127.0.0.1:46348 method = POST remote_addr = 127.0.0.1:41283 req_id = bf272e5e-cbe5-41d9-b505-34877ffe12d2 resource = Database result = Ok(()) uri = /v1/instances?project=utilization-test-project
40112025-04-30T06:43:53.852ZTRACtest_utilization (dropshot_external): loading roles actor = Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. } actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true local_addr = 127.0.0.1:46348 method = POST remote_addr = 127.0.0.1:41283 req_id = bf272e5e-cbe5-41d9-b505-34877ffe12d2 resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet uri = /v1/instances?project=utilization-test-project
40122025-04-30T06:43:53.852ZTRACtest_utilization (dropshot_external): authorize begin action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true local_addr = 127.0.0.1:46348 method = POST remote_addr = 127.0.0.1:41283 req_id = bf272e5e-cbe5-41d9-b505-34877ffe12d2 resource = Database uri = /v1/instances?project=utilization-test-project
40132025-04-30T06:43:53.852ZDEBGtest_utilization (dropshot_external): roles actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true local_addr = 127.0.0.1:46348 method = POST remote_addr = 127.0.0.1:41283 req_id = bf272e5e-cbe5-41d9-b505-34877ffe12d2 roles = RoleSet { roles: {} } uri = /v1/instances?project=utilization-test-project
40142025-04-30T06:43:53.853ZDEBGtest_utilization (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true local_addr = 127.0.0.1:46348 method = POST remote_addr = 127.0.0.1:41283 req_id = bf272e5e-cbe5-41d9-b505-34877ffe12d2 resource = Database result = Ok(()) uri = /v1/instances?project=utilization-test-project
40152025-04-30T06:43:53.860ZDEBGtest_utilization (dropshot_external): roles actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true local_addr = 127.0.0.1:46348 method = POST remote_addr = 127.0.0.1:41283 req_id = bf272e5e-cbe5-41d9-b505-34877ffe12d2 roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin"), (Silo, 001de000-5110-4000-8000-000000000000, "admin")} } uri = /v1/instances?project=utilization-test-project
40162025-04-30T06:43:53.864ZDEBGtest_utilization (dropshot_external): authorize result action = ListChildren actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true local_addr = 127.0.0.1:46348 method = POST remote_addr = 127.0.0.1:41283 req_id = bf272e5e-cbe5-41d9-b505-34877ffe12d2 resource = SiloUser { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000000, lookup_type: ById(001de000-5110-4000-8000-000000000000) }, key: 001de000-05e4-4000-8000-000000004007, lookup_type: ById(001de000-05e4-4000-8000-000000004007) } result = Ok(()) uri = /v1/instances?project=utilization-test-project
40172025-04-30T06:43:53.864ZTRACtest_utilization (dropshot_external): authorize begin action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true local_addr = 127.0.0.1:46348 method = POST remote_addr = 127.0.0.1:41283 req_id = bf272e5e-cbe5-41d9-b505-34877ffe12d2 resource = Database uri = /v1/instances?project=utilization-test-project
40182025-04-30T06:43:53.864ZDEBGtest_utilization (dropshot_external): roles actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true local_addr = 127.0.0.1:46348 method = POST remote_addr = 127.0.0.1:41283 req_id = bf272e5e-cbe5-41d9-b505-34877ffe12d2 roles = RoleSet { roles: {} } uri = /v1/instances?project=utilization-test-project
40192025-04-30T06:43:53.865ZDEBGtest_utilization (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true local_addr = 127.0.0.1:46348 method = POST remote_addr = 127.0.0.1:41283 req_id = bf272e5e-cbe5-41d9-b505-34877ffe12d2 resource = Database result = Ok(()) uri = /v1/instances?project=utilization-test-project
40202025-04-30T06:43:53.880ZTRACtest_utilization (dropshot_external): authorize begin action = ListChildren actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true local_addr = 127.0.0.1:46348 method = POST remote_addr = 127.0.0.1:41283 req_id = bf272e5e-cbe5-41d9-b505-34877ffe12d2 resource = Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000000, lookup_type: ById(001de000-5110-4000-8000-000000000000) }, key: 473b39f9-8669-45b3-b3d7-793f7e10fc31, lookup_type: ByName("utilization-test-project") } uri = /v1/instances?project=utilization-test-project
40212025-04-30T06:43:53.880ZTRACtest_utilization (dropshot_external): loading roles actor = Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. } actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true local_addr = 127.0.0.1:46348 method = POST remote_addr = 127.0.0.1:41283 req_id = bf272e5e-cbe5-41d9-b505-34877ffe12d2 resource_id = 473b39f9-8669-45b3-b3d7-793f7e10fc31 resource_type = Project uri = /v1/instances?project=utilization-test-project
40222025-04-30T06:43:53.880ZTRACtest_utilization (dropshot_external): authorize begin action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true local_addr = 127.0.0.1:46348 method = POST remote_addr = 127.0.0.1:41283 req_id = bf272e5e-cbe5-41d9-b505-34877ffe12d2 resource = Database uri = /v1/instances?project=utilization-test-project
40232025-04-30T06:43:53.880ZDEBGtest_utilization (dropshot_external): roles actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true local_addr = 127.0.0.1:46348 method = POST remote_addr = 127.0.0.1:41283 req_id = bf272e5e-cbe5-41d9-b505-34877ffe12d2 roles = RoleSet { roles: {} } uri = /v1/instances?project=utilization-test-project
40242025-04-30T06:43:53.881ZDEBGtest_utilization (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true local_addr = 127.0.0.1:46348 method = POST remote_addr = 127.0.0.1:41283 req_id = bf272e5e-cbe5-41d9-b505-34877ffe12d2 resource = Database result = Ok(()) uri = /v1/instances?project=utilization-test-project
40252025-04-30T06:43:53.884ZTRACtest_utilization (dropshot_external): loading roles actor = Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. } actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true local_addr = 127.0.0.1:46348 method = POST remote_addr = 127.0.0.1:41283 req_id = bf272e5e-cbe5-41d9-b505-34877ffe12d2 resource_id = 001de000-5110-4000-8000-000000000000 resource_type = Silo uri = /v1/instances?project=utilization-test-project
40262025-04-30T06:43:53.884ZTRACtest_utilization (dropshot_external): authorize begin action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true local_addr = 127.0.0.1:46348 method = POST remote_addr = 127.0.0.1:41283 req_id = bf272e5e-cbe5-41d9-b505-34877ffe12d2 resource = Database uri = /v1/instances?project=utilization-test-project
40272025-04-30T06:43:53.884ZDEBGtest_utilization (dropshot_external): roles actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true local_addr = 127.0.0.1:46348 method = POST remote_addr = 127.0.0.1:41283 req_id = bf272e5e-cbe5-41d9-b505-34877ffe12d2 roles = RoleSet { roles: {} } uri = /v1/instances?project=utilization-test-project
40282025-04-30T06:43:53.885ZDEBGtest_utilization (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true local_addr = 127.0.0.1:46348 method = POST remote_addr = 127.0.0.1:41283 req_id = bf272e5e-cbe5-41d9-b505-34877ffe12d2 resource = Database result = Ok(()) uri = /v1/instances?project=utilization-test-project
40292025-04-30T06:43:53.888ZTRACtest_utilization (dropshot_external): loading roles actor = Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. } actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true local_addr = 127.0.0.1:46348 method = POST remote_addr = 127.0.0.1:41283 req_id = bf272e5e-cbe5-41d9-b505-34877ffe12d2 resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet uri = /v1/instances?project=utilization-test-project
40302025-04-30T06:43:53.888ZTRACtest_utilization (dropshot_external): authorize begin action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true local_addr = 127.0.0.1:46348 method = POST remote_addr = 127.0.0.1:41283 req_id = bf272e5e-cbe5-41d9-b505-34877ffe12d2 resource = Database uri = /v1/instances?project=utilization-test-project
40312025-04-30T06:43:53.888ZDEBGtest_utilization (dropshot_external): roles actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true local_addr = 127.0.0.1:46348 method = POST remote_addr = 127.0.0.1:41283 req_id = bf272e5e-cbe5-41d9-b505-34877ffe12d2 roles = RoleSet { roles: {} } uri = /v1/instances?project=utilization-test-project
40322025-04-30T06:43:53.889ZDEBGtest_utilization (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true local_addr = 127.0.0.1:46348 method = POST remote_addr = 127.0.0.1:41283 req_id = bf272e5e-cbe5-41d9-b505-34877ffe12d2 resource = Database result = Ok(()) uri = /v1/instances?project=utilization-test-project
40332025-04-30T06:43:53.892ZDEBGtest_utilization (dropshot_external): roles actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true local_addr = 127.0.0.1:46348 method = POST remote_addr = 127.0.0.1:41283 req_id = bf272e5e-cbe5-41d9-b505-34877ffe12d2 roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin"), (Silo, 001de000-5110-4000-8000-000000000000, "admin")} } uri = /v1/instances?project=utilization-test-project
40342025-04-30T06:43:53.900ZDEBGtest_utilization (dropshot_external): authorize result action = ListChildren actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true local_addr = 127.0.0.1:46348 method = POST remote_addr = 127.0.0.1:41283 req_id = bf272e5e-cbe5-41d9-b505-34877ffe12d2 resource = Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000000, lookup_type: ById(001de000-5110-4000-8000-000000000000) }, key: 473b39f9-8669-45b3-b3d7-793f7e10fc31, lookup_type: ByName("utilization-test-project") } result = Ok(()) uri = /v1/instances?project=utilization-test-project
40352025-04-30T06:43:53.900ZTRACtest_utilization (dropshot_external): authorize begin action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true local_addr = 127.0.0.1:46348 method = POST remote_addr = 127.0.0.1:41283 req_id = bf272e5e-cbe5-41d9-b505-34877ffe12d2 resource = Database uri = /v1/instances?project=utilization-test-project
40362025-04-30T06:43:53.900ZDEBGtest_utilization (dropshot_external): roles actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true local_addr = 127.0.0.1:46348 method = POST remote_addr = 127.0.0.1:41283 req_id = bf272e5e-cbe5-41d9-b505-34877ffe12d2 roles = RoleSet { roles: {} } uri = /v1/instances?project=utilization-test-project
40372025-04-30T06:43:53.901ZDEBGtest_utilization (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true local_addr = 127.0.0.1:46348 method = POST remote_addr = 127.0.0.1:41283 req_id = bf272e5e-cbe5-41d9-b505-34877ffe12d2 resource = Database result = Ok(()) uri = /v1/instances?project=utilization-test-project
40382025-04-30T06:43:53.912ZTRACtest_utilization (InstanceAllocator): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000004ead, .. }) resource = Fleet
40392025-04-30T06:43:53.912ZTRACtest_utilization (InstanceAllocator): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000004ead, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
40402025-04-30T06:43:53.912ZTRACtest_utilization (InstanceAllocator): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000004ead, .. }) resource = Database
40412025-04-30T06:43:53.912ZDEBGtest_utilization (InstanceAllocator): roles roles = RoleSet { roles: {} }
40422025-04-30T06:43:53.919ZDEBGtest_utilization (InstanceAllocator): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000004ead, .. }) resource = Database result = Ok(())
40432025-04-30T06:43:53.925ZDEBGtest_utilization (InstanceAllocator): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "viewer")} }
40442025-04-30T06:43:53.929ZDEBGtest_utilization (InstanceAllocator): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000004ead, .. }) resource = Fleet result = Ok(())
40452025-04-30T06:43:53.929ZTRACtest_utilization (InstanceAllocator): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000004ead, .. }) resource = Database
40462025-04-30T06:43:53.929ZDEBGtest_utilization (InstanceAllocator): roles roles = RoleSet { roles: {} }
40472025-04-30T06:43:53.930ZDEBGtest_utilization (InstanceAllocator): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000004ead, .. }) resource = Database result = Ok(())
40482025-04-30T06:43:53.949ZINFOtest_utilization (SagaExecutor): preparing saga saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c saga_name = instance-create
40492025-04-30T06:43:53.952ZINFOtest_utilization (SEC): saga create dag = {"end_node":168,"graph":{"edge_property":"directed","edges":[[0,1,null],[1,2,null],[2,3,null],[3,4,null],[4,5,null],[5,6,null],[6,7,null],[7,8,null],[8,9,null],[9,10,null],[10,11,null],[11,12,null],[12,13,null],[13,14,null],[14,15,null],[15,16,null],[16,17,null],[17,18,null],[18,19,null],[19,20,null],[20,21,null],[21,22,null],[22,23,null],[23,24,null],[24,25,null],[25,26,null],[26,27,null],[27,28,null],[28,29,null],[29,30,null],[30,31,null],[31,32,null],[32,33,null],[33,34,null],[34,35,null],[35,36,null],[36,37,null],[37,38,null],[38,39,null],[39,40,null],[40,41,null],[41,42,null],[42,43,null],[43,44,null],[44,45,null],[45,46,null],[46,47,null],[47,48,null],[48,49,null],[49,50,null],[50,51,null],[51,52,null],[52,53,null],[53,54,null],[54,55,null],[55,56,null],[56,57,null],[57,58,null],[58,59,null],[59,60,null],[60,61,null],[61,62,null],[62,63,null],[63,64,null],[64,65,null],[65,66,null],[66,67,null],[67,68,null],[68,69,null],[69,70,null],[70,71,null],[71,72,null],[72,73,null],[73,74,null],[74,75,null],[75,76,null],[76,77,null],[77,78,null],[78,79,null],[79,80,null],[80,81,null],[81,82,null],[82,83,null],[83,84,null],[84,85,null],[85,86,null],[86,87,null],[87,88,null],[88,89,null],[89,90,null],[90,91,null],[91,92,null],[92,93,null],[93,94,null],[94,95,null],[95,96,null],[96,97,null],[97,98,null],[98,99,null],[99,100,null],[100,101,null],[101,102,null],[102,103,null],[103,104,null],[104,105,null],[105,106,null],[106,107,null],[107,108,null],[108,109,null],[109,110,null],[110,111,null],[111,112,null],[112,113,null],[113,114,null],[114,115,null],[115,116,null],[116,117,null],[117,118,null],[118,119,null],[119,120,null],[120,121,null],[121,122,null],[122,123,null],[123,124,null],[124,125,null],[125,126,null],[126,127,null],[127,128,null],[128,129,null],[129,130,null],[130,131,null],[131,132,null],[132,133,null],[133,134,null],[134,135,null],[135,136,null],[136,137,null],[137,138,null],[138,139,null],[139,140,null],[140,141,null],[141,142,null],[142,143,null],[143,144,null],[144,145,null],[145,146,null],[146,147,null],[147,148,null],[148,149,null],[149,150,null],[150,151,null],[151,152,null],[152,153,null],[153,154,null],[154,155,null],[155,156,null],[156,157,null],[157,158,null],[158,159,null],[159,160,null],[160,161,null],[161,162,null],[162,163,null],[163,164,null],[164,165,null],[165,166,null],[167,0,null],[166,168,null]],"node_holes":[],"nodes":[{"Constant":{"name":"instance_id","value":"d7bd2deb-ee27-4661-9e9a-524d734cd6ec"}},{"Action":{"action_name":"instance_create.create_instance_record","label":"CreateInstanceRecord","name":"instance_record"}},{"Action":{"action_name":"instance_create.associate_ssh_keys","label":"AssociateSshKeys","name":"output"}},{"Constant":{"name":"network_interface_params0","value":{"instance_id":"d7bd2deb-ee27-4661-9e9a-524d734cd6ec","new_id":"64c48731-6cc1-46ff-99f2-86f6c481d0c6","saga_params":{"boundary_switches":[],"create_params":{"anti_affinity_groups":[],"auto_restart_policy":null,"boot_disk":null,"description":"instance \\"utilization-test-instance\\"","disks":[],"external_ips":[],"hostname":"the-host","memory":1073741824,"name":"utilization-test-instance","ncpus":4,"network_interfaces":{"type":"default"},"ssh_public_keys":[],"start":true,"user_data":"I2Nsb3VkLWNvbmZpZwpzeXN0ZW1faW5mbzoKICBkZWZhdWx0X3VzZXI6CiAgICBuYW1lOiBveGlkZQ=="},"project_id":"473b39f9-8669-45b3-b3d7-793f7e10fc31","serialized_authn":{"kind":{"Authenticated":[{"actor":{"SiloUser":{"silo_id":"001de000-5110-4000-8000-000000000000","silo_user_id":"001de000-05e4-4000-8000-000000004007"}}},{"mapped_fleet_roles":{}}]}}},"which":0}}},{"SubsagaStart":{"params_node_name":"network_interface_params0","saga_name":"instance-create-nic0"}},{"Action":{"action_name":"instance_create.create_network_interface","label":"CreateNetworkInterface0","name":"output"}},{"SubsagaEnd":{"name":"network_interface0"}},{"Constant":{"name":"network_interface_params1","value":{"instance_id":"d7bd2deb-ee27-4661-9e9a-524d734cd6ec","new_id":"79be82ce-957c-439a-b67e-f1dacdb57f1e","saga_params":{"boundary_switches":[],"create_params":{"anti_affinity_groups":[],"auto_restart_policy":null,"boot_disk":null,"description":"instance \\"utilization-test-instance\\"","disks":[],"external_ips":[],"hostname":"the-host","memory":1073741824,"name":"utilization-test-instance","ncpus":4,"network_interfaces":{"type":"default"},"ssh_public_keys":[],"start":true,"user_data":"I2Nsb3VkLWNvbmZpZwpzeXN0ZW1faW5mbzoKICBkZWZhdWx0X3VzZXI6CiAgICBuYW1lOiBveGlkZQ=="},"project_id":"473b39f9-8669-45b3-b3d7-793f7e10fc31","serialized_authn":{"kind":{"Authenticated":[{"actor":{"SiloUser":{"silo_id":"001de000-5110-4000-8000-000000000000","silo_user_id":"001de000-05e4-4000-8000-000000004007"}}},{"mapped_fleet_roles":{}}]}}},"which":1}}},{"SubsagaStart":{"params_node_name":"network_interface_params1","saga_name":"instance-create-nic1"}},{"Action":{"action_name":"instance_create.create_network_interface","label":"CreateNetworkInterface1","name":"output"}},{"SubsagaEnd":{"name":"network_interface1"}},{"Constant":{"name":"network_interface_params2","value":{"instance_id":"d7bd2deb-ee27-4661-9e9a-524d734cd6ec","new_id":"628c4c7e-b430-4a2a-bfd4-c1a8a5a74fa6","saga_params":{"boundary_switches":[],"create_params":{"anti_affinity_groups":[],"auto_restart_policy":null,"boot_disk":null,"description":"instance \\"utilization-test-instance\\"","disks":[],"external_ips":[],"hostname":"the-host","memory":1073741824,"name":"utilization-test-instance","ncpus":4,"network_interfaces":{"type":"default"},"ssh_public_keys":[],"start":true,"user_data":"I2Nsb3VkLWNvbmZpZwpzeXN0ZW1faW5mbzoKICBkZWZhdWx0X3VzZXI6CiAgICBuYW1lOiBveGlkZQ=="},"project_id":"473b39f9-8669-45b3-b3d7-793f7e10fc31","serialized_authn":{"kind":{"Authenticated":[{"actor":{"SiloUser":{"silo_id":"001de000-5110-4000-8000-000000000000","silo_user_id":"001de000-05e4-4000-8000-000000004007"}}},{"mapped_fleet_roles":{}}]}}},"which":2}}},{"SubsagaStart":{"params_node_name":"network_interface_params2","saga_name":"instance-create-nic2"}},{"Action":{"action_name":"instance_create.create_network_interface","label":"CreateNetworkInterface2","name":"output"}},{"SubsagaEnd":{"name":"network_interface2"}},{"Constant":{"name":"network_interface_params3","value":{"instance_id":"d7bd2deb-ee27-4661-9e9a-524d734cd6ec","new_id":"9b0302b2-2547-4705-b879-b2d896fa654b","saga_params":{"boundary_switches":[],"create_params":{"anti_affinity_groups":[],"auto_restart_policy":null,"boot_disk":null,"description":"instance \\"utilization-test-instance\\"","disks":[],"external_ips":[],"hostname":"the-host","memory":1073741824,"name":"utilization-test-instance","ncpus":4,"network_interfaces":{"type":"default"},"ssh_public_keys":[],"start":true,"user_data":"I2Nsb3VkLWNvbmZpZwpzeXN0ZW1faW5mbzoKICBkZWZhdWx0X3VzZXI6CiAgICBuYW1lOiBveGlkZQ=="},"project_id":"473b39f9-8669-45b3-b3d7-793f7e10fc31","serialized_authn":{"kind":{"Authenticated":[{"actor":{"SiloUser":{"silo_id":"001de000-5110-4000-8000-000000000000","silo_user_id":"001de000-05e4-4000-8000-000000004007"}}},{"mapped_fleet_roles":{}}]}}},"which":3}}},{"SubsagaStart":{"params_node_name":"network_interface_params3","saga_name":"instance-create-nic3"}},{"Action":{"action_name":"instance_create.create_network_interface","label":"CreateNetworkInterface3","name":"output"}},{"SubsagaEnd":{"name":"network_interface3"}},{"Constant":{"name":"network_interface_params4","value":{"instance_id":"d7bd2deb-ee27-4661-9e9a-524d734cd6ec","new_id":"7390678b-5d44-43e6-b46d-3c25a5eebd71","saga_params":{"boundary_switches":[],"create_params":{"anti_affinity_groups":[],"auto_restart_policy":null,"boot_disk":null,"description":"instance \\"utilization-test-instance\\"","disks":[],"external_ips":[],"hostname":"the-host","memory":1073741824,"name":"utilization-test-instance","ncpus":4,"network_interfaces":{"type":"default"},"ssh_public_keys":[],"start":true,"user_data":"I2Nsb3VkLWNvbmZpZwpzeXN0ZW1faW5mbzoKICBkZWZhdWx0X3VzZXI6CiAgICBuYW1lOiBveGlkZQ=="},"project_id":"473b39f9-8669-45b3-b3d7-793f7e10fc31","serialized_authn":{"kind":{"Authenticated":[{"actor":{"SiloUser":{"silo_id":"001de000-5110-4000-8000-000000000000","silo_user_id":"001de000-05e4-4000-8000-000000004007"}}},{"mapped_fleet_roles":{}}]}}},"which":4}}},{"SubsagaStart":{"params_node_name":"network_interface_params4","saga_name":"instance-create-nic4"}},{"Action":{"action_name":"instance_create.create_network_interface","label":"CreateNetworkInterface4","name":"output"}},{"SubsagaEnd":{"name":"network_interface4"}},{"Constant":{"name":"network_interface_params5","value":{"instance_id":"d7bd2deb-ee27-4661-9e9a-524d734cd6ec","new_id":"2a09f6d6-3de1-4bea-b89c-6fc00eaf504e","saga_params":{"boundary_switches":[],"create_params":{"anti_affinity_groups":[],"auto_restart_policy":null,"boot_disk":null,"description":"instance \\"utilization-test-instance\\"","disks":[],"external_ips":[],"hostname":"the-host","memory":1073741824,"name":"utilization-test-instance","ncpus":4,"network_interfaces":{"type":"default"},"ssh_public_keys":[],"start":true,"user_data":"I2Nsb3VkLWNvbmZpZwpzeXN0ZW1faW5mbzoKICBkZWZhdWx0X3VzZXI6CiAgICBuYW1lOiBveGlkZQ=="},"project_id":"473b39f9-8669-45b3-b3d7-793f7e10fc31","serialized_authn":{"kind":{"Authenticated":[{"actor":{"SiloUser":{"silo_id":"001de000-5110-4000-8000-000000000000","silo_user_id":"001de000-05e4-4000-8000-000000004007"}}},{"mapped_fleet_roles":{}}]}}},"which":5}}},{"SubsagaStart":{"params_node_name":"network_interface_params5","saga_name":"instance-create-nic5"}},{"Action":{"action_name":"instance_create.create_network_interface","label":"CreateNetworkInterface5","name":"output"}},{"SubsagaEnd":{"name":"network_interface5"}},{"Constant":{"name":"network_interface_params6","value":{"instance_id":"d7bd2deb-ee27-4661-9e9a-524d734cd6ec","new_id":"86338488-d577-4ce7-a835-4c7491bc0d88","saga_params":{"boundary_switches":[],"create_params":{"anti_affinity_groups":[],"auto_restart_policy":null,"boot_disk":null,"description":"instance \\"utilization-test-instance\\"","disks":[],"external_ips":[],"hostname":"the-host","memory":1073741824,"name":"utilization-test-instance","ncpus":4,"network_interfaces":{"type":"default"},"ssh_public_keys":[],"start":true,"user_data":"I2Nsb3VkLWNvbmZpZwpzeXN0ZW1faW5mbzoKICBkZWZhdWx0X3VzZXI6CiAgICBuYW1lOiBveGlkZQ=="},"project_id":"473b39f9-8669-45b3-b3d7-793f7e10fc31","serialized_authn":{"kind":{"Authenticated":[{"actor":{"SiloUser":{"silo_id":"001de000-5110-4000-8000-000000000000","silo_user_id":"001de000-05e4-4000-8000-000000004007"}}},{"mapped_fleet_roles":{}}]}}},"which":6}}},{"SubsagaStart":{"params_node_name":"network_interface_params6","saga_name":"instance-create-nic6"}},{"Action":{"action_name":"instance_create.create_network_interface","label":"CreateNetworkInterface6","name":"output"}},{"SubsagaEnd":{"name":"network_interface6"}},{"Constant":{"name":"network_interface_params7","value":{"instance_id":"d7bd2deb-ee27-4661-9e9a-524d734cd6ec","new_id":"d7156580-5c17-44d4-bbc0-c78d1b1133df","saga_params":{"boundary_switches":[],"create_params":{"anti_affinity_groups":[],"auto_restart_policy":null,"boot_disk":null,"description":"instance \\"utilization-test-instance\\"","disks":[],"external_ips":[],"hostname":"the-host","memory":1073741824,"name":"utilization-test-instance","ncpus":4,"network_interfaces":{"type":"default"},"ssh_public_keys":[],"start":true,"user_data":"I2Nsb3VkLWNvbmZpZwpzeXN0ZW1faW5mbzoKICBkZWZhdWx0X3VzZXI6CiAgICBuYW1lOiBveGlkZQ=="},"project_id":"473b39f9-8669-45b3-b3d7-793f7e10fc31","serialized_authn":{"kind":{"Authenticated":[{"actor":{"SiloUser":{"silo_id":"001de000-5110-4000-8000-000000000000","silo_user_id":"001de000-05e4-4000-8000-000000004007"}}},{"mapped_fleet_roles":{}}]}}},"which":7}}},{"SubsagaStart":{"params_node_name":"network_interface_params7","saga_name":"instance-create-nic7"}},{"Action":{"action_name":"instance_create.create_network_interface","label":"CreateNetworkInterface7","name":"output"}},{"SubsagaEnd":{"name":"network_interface7"}},{"Action":{"action_name":"common.uuid_generate","label":"CreateSnatIpId","name":"snat_ip_id"}},{"Action":{"action_name":"instance_create.create_snat_ip","label":"CreateSnatIp","name":"snat_ip"}},{"Constant":{"name":"external_ip_params0","value":{"instance_id":"d7bd2deb-ee27-4661-9e9a-524d734cd6ec","new_id":"d135d564-8cd2-4690-a1c9-1629e2d7c21e","saga_params":{"boundary_switches":[],"create_params":{"anti_affinity_groups":[],"auto_restart_policy":null,"boot_disk":null,"description":"instance \\"utilization-test-instance\\"","disks":[],"external_ips":[],"hostname":"the-host","memory":1073741824,"name":"utilization-test-instance","ncpus":4,"network_interfaces":{"type":"default"},"ssh_public_keys":[],"start":true,"user_data":"I2Nsb3VkLWNvbmZpZwpzeXN0ZW1faW5mbzoKICBkZWZhdWx0X3VzZXI6CiAgICBuYW1lOiBveGlkZQ=="},"project_id":"473b39f9-8669-45b3-b3d7-793f7e10fc31","serialized_authn":{"kind":{"Authenticated":[{"actor":{"SiloUser":{"silo_id":"001de000-5110-4000-8000-000000000000","silo_user_id":"001de000-05e4-4000-8000-000000004007"}}},{"mapped_fleet_roles":{}}]}}},"which":0}}},{"SubsagaStart":{"params_node_name":"external_ip_params0","saga_name":"instance-create-external-ip0"}},{"Action":{"action_name":"instance_create.create_external_ip","label":"CreateExternalIp0","name":"external-ip-0"}},{"SubsagaEnd":{"name":"external_ip0"}},{"Constant":{"name":"external_ip_params1","value":{"instance_id":"d7bd2deb-ee27-4661-9e9a-524d734cd6ec","new_id":"60556df4-fdfb-48b6-8878-e6da4a1201e5","saga_params":{"boundary_switches":[],"create_params":{"anti_affinity_groups":[],"auto_restart_policy":null,"boot_disk":null,"description":"instance \\"utilization-test-instance\\"","disks":[],"external_ips":[],"hostname":"the-host","memory":1073741824,"name":"utilization-test-instance","ncpus":4,"network_interfaces":{"type":"default"},"ssh_public_keys":[],"start":true,"user_data":"I2Nsb3VkLWNvbmZpZwpzeXN0ZW1faW5mbzoKICBkZWZhdWx0X3VzZXI6CiAgICBuYW1lOiBveGlkZQ=="},"project_id":"473b39f9-8669-45b3-b3d7-793f7e10fc31","serialized_authn":{"kind":{"Authenticated":[{"actor":{"SiloUser":{"silo_id":"001de000-5110-4000-8000-000000000000","silo_user_id":"001de000-05e4-4000-8000-000000004007"}}},{"mapped_fleet_roles":{}}]}}},"which":1}}},{"SubsagaStart":{"params_node_name":"external_ip_params1","saga_name":"instance-create-external-ip1"}},{"Action":{"action_name":"instance_create.create_external_ip","label":"CreateExternalIp1","name":"external-ip-1"}},{"SubsagaEnd":{"name":"external_ip1"}},{"Constant":{"name":"external_ip_params2","value":{"instance_id":"d7bd2deb-ee27-4661-9e9a-524d734cd6ec","new_id":"95e2067e-7259-4904-bcc6-199c85e45b17","saga_params":{"boundary_switches":[],"create_params":{"anti_affinity_groups":[],"auto_restart_policy":null,"boot_disk":null,"description":"instance \\"utilization-test-instance\\"","disks":[],"external_ips":[],"hostname":"the-host","memory":1073741824,"name":"utilization-test-instance","ncpus":4,"network_interfaces":{"type":"default"},"ssh_public_keys":[],"start":true,"user_data":"I2Nsb3VkLWNvbmZpZwpzeXN0ZW1faW5mbzoKICBkZWZhdWx0X3VzZXI6CiAgICBuYW1lOiBveGlkZQ=="},"project_id":"473b39f9-8669-45b3-b3d7-793f7e10fc31","serialized_authn":{"kind":{"Authenticated":[{"actor":{"SiloUser":{"silo_id":"001de000-5110-4000-8000-000000000000","silo_user_id":"001de000-05e4-4000-8000-000000004007"}}},{"mapped_fleet_roles":{}}]}}},"which":2}}},{"SubsagaStart":{"params_node_name":"external_ip_params2","saga_name":"instance-create-external-ip2"}},{"Action":{"action_name":"instance_create.create_external_ip","label":"CreateExternalIp2","name":"external-ip-2"}},{"SubsagaEnd":{"name":"external_ip2"}},{"Constant":{"name":"external_ip_params3","value":{"instance_id":"d7bd2deb-ee27-4661-9e9a-524d734cd6ec","new_id":"fdb5b695-fb92-422f-b187-3e8141c4416b","saga_params":{"boundary_switches":[],"create_params":{"anti_affinity_groups":[],"auto_restart_policy":null,"boot_disk":null,"description":"instance \\"utilization-test-instance\\"","disks":[],"external_ips":[],"hostname":"the-host","memory":1073741824,"name":"utilization-test-instance","ncpus":4,"network_interfaces":{"type":"default"},"ssh_public_keys":[],"start":true,"user_data":"I2Nsb3VkLWNvbmZpZwpzeXN0ZW1faW5mbzoKICBkZWZhdWx0X3VzZXI6CiAgICBuYW1lOiBveGlkZQ=="},"project_id":"473b39f9-8669-45b3-b3d7-793f7e10fc31","serialized_authn":{"kind":{"Authenticated":[{"actor":{"SiloUser":{"silo_id":"001de000-5110-4000-8000-000000000000","silo_user_id":"001de000-05e4-4000-8000-000000004007"}}},{"mapped_fleet_roles":{}}]}}},"which":3}}},{"SubsagaStart":{"params_node_name":"external_ip_params3","saga_name":"instance-create-external-ip3"}},{"Action":{"action_name":"instance_create.create_external_ip","label":"CreateExternalIp3","name":"external-ip-3"}},{"SubsagaEnd":{"name":"external_ip3"}},{"Constant":{"name":"external_ip_params4","value":{"instance_id":"d7bd2deb-ee27-4661-9e9a-524d734cd6ec","new_id":"1db204ac-45f7-4301-b4bb-8b82e9fce5f5","saga_params":{"boundary_switches":[],"create_params":{"anti_affinity_groups":[],"auto_restart_policy":null,"boot_disk":null,"description":"instance \\"utilization-test-instance\\"","disks":[],"external_ips":[],"hostname":"the-host","memory":1073741824,"name":"utilization-test-instance","ncpus":4,"network_interfaces":{"type":"default"},"ssh_public_keys":[],"start":true,"user_data":"I2Nsb3VkLWNvbmZpZwpzeXN0ZW1faW5mbzoKICBkZWZhdWx0X3VzZXI6CiAgICBuYW1lOiBveGlkZQ=="},"project_id":"473b39f9-8669-45b3-b3d7-793f7e10fc31","serialized_authn":{"kind":{"Authenticated":[{"actor":{"SiloUser":{"silo_id":"001de000-5110-4000-8000-000000000000","silo_user_id":"001de000-05e4-4000-8000-000000004007"}}},{"mapped_fleet_roles":{}}]}}},"which":4}}},{"SubsagaStart":{"params_node_name":"external_ip_params4","saga_name":"instance-create-external-ip4"}},{"Action":{"action_name":"instance_create.create_external_ip","label":"CreateExternalIp4","name":"external-ip-4"}},{"SubsagaEnd":{"name":"external_ip4"}},{"Constant":{"name":"external_ip_params5","value":{"instance_id":"d7bd2deb-ee27-4661-9e9a-524d734cd6ec","new_id":"caf1be44-80eb-4e28-91d3-8f81c65992f5","saga_params":{"boundary_switches":[],"create_params":{"anti_affinity_groups":[],"auto_restart_policy":null,"boot_disk":null,"description":"instance \\"utilization-test-instance\\"","disks":[],"external_ips":[],"hostname":"the-host","memory":1073741824,"name":"utilization-test-instance","ncpus":4,"network_interfaces":{"type":"default"},"ssh_public_keys":[],"start":true,"user_data":"I2Nsb3VkLWNvbmZpZwpzeXN0ZW1faW5mbzoKICBkZWZhdWx0X3VzZXI6CiAgICBuYW1lOiBveGlkZQ=="},"project_id":"473b39f9-8669-45b3-b3d7-793f7e10fc31","serialized_authn":{"kind":{"Authenticated":[{"actor":{"SiloUser":{"silo_id":"001de000-5110-4000-8000-000000000000","silo_user_id":"001de000-05e4-4000-8000-000000004007"}}},{"mapped_fleet_roles":{}}]}}},"which":5}}},{"SubsagaStart":{"params_node_name":"external_ip_params5","saga_name":"instance-create-external-ip5"}},{"Action":{"action_name":"instance_create.create_external_ip","label":"CreateExternalIp5","name":"external-ip-5"}},{"SubsagaEnd":{"name":"external_ip5"}},{"Constant":{"name":"external_ip_params6","value":{"instance_id":"d7bd2deb-ee27-4661-9e9a-524d734cd6ec","new_id":"c7469230-9dc6-409d-93da-9629008d5d16","saga_params":{"boundary_switches":[],"create_params":{"anti_affinity_groups":[],"auto_restart_policy":null,"boot_disk":null,"description":"instance \\"utilization-test-instance\\"","disks":[],"external_ips":[],"hostname":"the-host","memory":1073741824,"name":"utilization-test-instance","ncpus":4,"network_interfaces":{"type":"default"},"ssh_public_keys":[],"start":true,"user_data":"I2Nsb3VkLWNvbmZpZwpzeXN0ZW1faW5mbzoKICBkZWZhdWx0X3VzZXI6CiAgICBuYW1lOiBveGlkZQ=="},"project_id":"473b39f9-8669-45b3-b3d7-793f7e10fc31","serialized_authn":{"kind":{"Authenticated":[{"actor":{"SiloUser":{"silo_id":"001de000-5110-4000-8000-000000000000","silo_user_id":"001de000-05e4-4000-8000-000000004007"}}},{"mapped_fleet_roles":{}}]}}},"which":6}}},{"SubsagaStart":{"params_node_name":"external_ip_params6","saga_name":"instance-create-external-ip6"}},{"Action":{"action_name":"instance_create.create_external_ip","label":"CreateExternalIp6","name":"external-ip-6"}},{"SubsagaEnd":{"name":"external_ip6"}},{"Constant":{"name":"external_ip_params7","value":{"instance_id":"d7bd2deb-ee27-4661-9e9a-524d734cd6ec","new_id":"1943d0ee-055e-4aa1-a7a8-403496b4dbcb","saga_params":{"boundary_switches":[],"create_params":{"anti_affinity_groups":[],"auto_restart_policy":null,"boot_disk":null,"description":"instance \\"utilization-test-instance\\"","disks":[],"external_ips":[],"hostname":"the-host","memory":1073741824,"name":"utilization-test-instance","ncpus":4,"network_interfaces":{"type":"default"},"ssh_public_keys":[],"start":true,"user_data":"I2Nsb3VkLWNvbmZpZwpzeXN0ZW1faW5mbzoKICBkZWZhdWx0X3VzZXI6CiAgICBuYW1lOiBveGlkZQ=="},"project_id":"473b39f9-8669-45b3-b3d7-793f7e10fc31","serialized_authn":{"kind":{"Authenticated":[{"actor":{"SiloUser":{"silo_id":"001de000-5110-4000-8000-000000000000","silo_user_id":"001de000-05e4-4000-8000-000000004007"}}},{"mapped_fleet_roles":{}}]}}},"which":7}}},{"SubsagaStart":{"params_node_name":"external_ip_params7","saga_name":"instance-create-external-ip7"}},{"Action":{"action_name":"instance_create.create_external_ip","label":"CreateExternalIp7","name":"external-ip-7"}},{"SubsagaEnd":{"name":"external_ip7"}},{"Constant":{"name":"external_ip_params8","value":{"instance_id":"d7bd2deb-ee27-4661-9e9a-524d734cd6ec","new_id":"e9418c49-b114-4554-82a3-43f65092bf41","saga_params":{"boundary_switches":[],"create_params":{"anti_affinity_groups":[],"auto_restart_policy":null,"boot_disk":null,"description":"instance \\"utilization-test-instance\\"","disks":[],"external_ips":[],"hostname":"the-host","memory":1073741824,"name":"utilization-test-instance","ncpus":4,"network_interfaces":{"type":"default"},"ssh_public_keys":[],"start":true,"user_data":"I2Nsb3VkLWNvbmZpZwpzeXN0ZW1faW5mbzoKICBkZWZhdWx0X3VzZXI6CiAgICBuYW1lOiBveGlkZQ=="},"project_id":"473b39f9-8669-45b3-b3d7-793f7e10fc31","serialized_authn":{"kind":{"Authenticated":[{"actor":{"SiloUser":{"silo_id":"001de000-5110-4000-8000-000000000000","silo_user_id":"001de000-05e4-4000-8000-000000004007"}}},{"mapped_fleet_roles":{}}]}}},"which":8}}},{"SubsagaStart":{"params_node_name":"external_ip_params8","saga_name":"instance-create-external-ip8"}},{"Action":{"action_name":"instance_create.create_external_ip","label":"CreateExternalIp8","name":"external-ip-8"}},{"SubsagaEnd":{"name":"external_ip8"}},{"Constant":{"name":"external_ip_params9","value":{"instance_id":"d7bd2deb-ee27-4661-9e9a-524d734cd6ec","new_id":"44dd706b-a60c-43ba-9fee-2f47969c2494","saga_params":{"boundary_switches":[],"create_params":{"anti_affinity_groups":[],"auto_restart_policy":null,"boot_disk":null,"description":"instance \\"utilization-test-instance\\"","disks":[],"external_ips":[],"hostname":"the-host","memory":1073741824,"name":"utilization-test-instance","ncpus":4,"network_interfaces":{"type":"default"},"ssh_public_keys":[],"start":true,"user_data":"I2Nsb3VkLWNvbmZpZwpzeXN0ZW1faW5mbzoKICBkZWZhdWx0X3VzZXI6CiAgICBuYW1lOiBveGlkZQ=="},"project_id":"473b39f9-8669-45b3-b3d7-793f7e10fc31","serialized_authn":{"kind":{"Authenticated":[{"actor":{"SiloUser":{"silo_id":"001de000-5110-4000-8000-000000000000","silo_user_id":"001de000-05e4-4000-8000-000000004007"}}},{"mapped_fleet_roles":{}}]}}},"which":9}}},{"SubsagaStart":{"params_node_name":"external_ip_params9","saga_name":"instance-create-external-ip9"}},{"Action":{"action_name":"instance_create.create_external_ip","label":"CreateExternalIp9","name":"external-ip-9"}},{"SubsagaEnd":{"name":"external_ip9"}},{"Constant":{"name":"external_ip_params10","value":{"instance_id":"d7bd2deb-ee27-4661-9e9a-524d734cd6ec","new_id":"22abc147-674c-4b2d-aaaa-f1b4e56bec0d","saga_params":{"boundary_switches":[],"create_params":{"anti_affinity_groups":[],"auto_restart_policy":null,"boot_disk":null,"description":"instance \\"utilization-test-instance\\"","disks":[],"external_ips":[],"hostname":"the-host","memory":1073741824,"name":"utilization-test-instance","ncpus":4,"network_interfaces":{"type":"default"},"ssh_public_keys":[],"start":true,"user_data":"I2Nsb3VkLWNvbmZpZwpzeXN0ZW1faW5mbzoKICBkZWZhdWx0X3VzZXI6CiAgICBuYW1lOiBveGlkZQ=="},"project_id":"473b39f9-8669-45b3-b3d7-793f7e10fc31","serialized_authn":{"kind":{"Authenticated":[{"actor":{"SiloUser":{"silo_id":"001de000-5110-4000-8000-000000000000","silo_user_id":"001de000-05e4-4000-8000-000000004007"}}},{"mapped_fleet_roles":{}}]}}},"which":10}}},{"SubsagaStart":{"params_node_name":"external_ip_params10","saga_name":"instance-create-external-ip10"}},{"Action":{"action_name":"instance_create.create_external_ip","label":"CreateExternalIp10","name":"external-ip-10"}},{"SubsagaEnd":{"name":"external_ip10"}},{"Constant":{"name":"external_ip_params11","value":{"instance_id":"d7bd2deb-ee27-4661-9e9a-524d734cd6ec","new_id":"90899943-1eb7-4a82-93b8-3ca4311fc81e","saga_params":{"boundary_switches":[],"create_params":{"anti_affinity_groups":[],"auto_restart_policy":null,"boot_disk":null,"description":"instance \\"utilization-test-instance\\"","disks":[],"external_ips":[],"hostname":"the-host","memory":1073741824,"name":"utilization-test-instance","ncpus":4,"network_interfaces":{"type":"default"},"ssh_public_keys":[],"start":true,"user_data":"I2Nsb3VkLWNvbmZpZwpzeXN0ZW1faW5mbzoKICBkZWZhdWx0X3VzZXI6CiAgICBuYW1lOiBveGlkZQ=="},"project_id":"473b39f9-8669-45b3-b3d7-793f7e10fc31","serialized_authn":{"kind":{"Authenticated":[{"actor":{"SiloUser":{"silo_id":"001de000-5110-4000-8000-000000000000","silo_user_id":"001de000-05e4-4000-8000-000000004007"}}},{"mapped_fleet_roles":{}}]}}},"which":11}}},{"SubsagaStart":{"params_node_name":"external_ip_params11","saga_name":"instance-create-external-ip11"}},{"Action":{"action_name":"instance_create.create_external_ip","label":"CreateExternalIp11","name":"external-ip-11"}},{"SubsagaEnd":{"name":"external_ip11"}},{"Constant":{"name":"external_ip_params12","value":{"instance_id":"d7bd2deb-ee27-4661-9e9a-524d734cd6ec","new_id":"6979ef71-cbe0-4065-8900-ae9ef8686f28","saga_params":{"boundary_switches":[],"create_params":{"anti_affinity_groups":[],"auto_restart_policy":null,"boot_disk":null,"description":"instance \\"utilization-test-instance\\"","disks":[],"external_ips":[],"hostname":"the-host","memory":1073741824,"name":"utilization-test-instance","ncpus":4,"network_interfaces":{"type":"default"},"ssh_public_keys":[],"start":true,"user_data":"I2Nsb3VkLWNvbmZpZwpzeXN0ZW1faW5mbzoKICBkZWZhdWx0X3VzZXI6CiAgICBuYW1lOiBveGlkZQ=="},"project_id":"473b39f9-8669-45b3-b3d7-793f7e10fc31","serialized_authn":{"kind":{"Authenticated":[{"actor":{"SiloUser":{"silo_id":"001de000-5110-4000-8000-000000000000","silo_user_id":"001de000-05e4-4000-8000-000000004007"}}},{"mapped_fleet_roles":{}}]}}},"which":12}}},{"SubsagaStart":{"params_node_name":"external_ip_params12","saga_name":"instance-create-external-ip12"}},{"Action":{"action_name":"instance_create.create_external_ip","label":"CreateExternalIp12","name":"external-ip-12"}},{"SubsagaEnd":{"name":"external_ip12"}},{"Constant":{"name":"external_ip_params13","value":{"instance_id":"d7bd2deb-ee27-4661-9e9a-524d734cd6ec","new_id":"5e5d8ca1-846b-484c-a5a7-5cc20d5140d3","saga_params":{"boundary_switches":[],"create_params":{"anti_affinity_groups":[],"auto_restart_policy":null,"boot_disk":null,"description":"instance \\"utilization-test-instance\\"","disks":[],"external_ips":[],"hostname":"the-host","memory":1073741824,"name":"utilization-test-instance","ncpus":4,"network_interfaces":{"type":"default"},"ssh_public_keys":[],"start":true,"user_data":"I2Nsb3VkLWNvbmZpZwpzeXN0ZW1faW5mbzoKICBkZWZhdWx0X3VzZXI6CiAgICBuYW1lOiBveGlkZQ=="},"project_id":"473b39f9-8669-45b3-b3d7-793f7e10fc31","serialized_authn":{"kind":{"Authenticated":[{"actor":{"SiloUser":{"silo_id":"001de000-5110-4000-8000-000000000000","silo_user_id":"001de000-05e4-4000-8000-000000004007"}}},{"mapped_fleet_roles":{}}]}}},"which":13}}},{"SubsagaStart":{"params_node_name":"external_ip_params13","saga_name":"instance-create-external-ip13"}},{"Action":{"action_name":"instance_create.create_external_ip","label":"CreateExternalIp13","name":"external-ip-13"}},{"SubsagaEnd":{"name":"external_ip13"}},{"Constant":{"name":"external_ip_params14","value":{"instance_id":"d7bd2deb-ee27-4661-9e9a-524d734cd6ec","new_id":"f0b5c727-87b5-41de-aede-b1bc7b951264","saga_params":{"boundary_switches":[],"create_params":{"anti_affinity_groups":[],"auto_restart_policy":null,"boot_disk":null,"description":"instance \\"utilization-test-instance\\"","disks":[],"external_ips":[],"hostname":"the-host","memory":1073741824,"name":"utilization-test-instance","ncpus":4,"network_interfaces":{"type":"default"},"ssh_public_keys":[],"start":true,"user_data":"I2Nsb3VkLWNvbmZpZwpzeXN0ZW1faW5mbzoKICBkZWZhdWx0X3VzZXI6CiAgICBuYW1lOiBveGlkZQ=="},"project_id":"473b39f9-8669-45b3-b3d7-793f7e10fc31","serialized_authn":{"kind":{"Authenticated":[{"actor":{"SiloUser":{"silo_id":"001de000-5110-4000-8000-000000000000","silo_user_id":"001de000-05e4-4000-8000-000000004007"}}},{"mapped_fleet_roles":{}}]}}},"which":14}}},{"SubsagaStart":{"params_node_name":"external_ip_params14","saga_name":"instance-create-external-ip14"}},{"Action":{"action_name":"instance_create.create_external_ip","label":"CreateExternalIp14","name":"external-ip-14"}},{"SubsagaEnd":{"name":"external_ip14"}},{"Constant":{"name":"external_ip_params15","value":{"instance_id":"d7bd2deb-ee27-4661-9e9a-524d734cd6ec","new_id":"1c2a0f63-abc4-4eef-b4b2-335cad832ba3","saga_params":{"boundary_switches":[],"create_params":{"anti_affinity_groups":[],"auto_restart_policy":null,"boot_disk":null,"description":"instance \\"utilization-test-instance\\"","disks":[],"external_ips":[],"hostname":"the-host","memory":1073741824,"name":"utilization-test-instance","ncpus":4,"network_interfaces":{"type":"default"},"ssh_public_keys":[],"start":true,"user_data":"I2Nsb3VkLWNvbmZpZwpzeXN0ZW1faW5mbzoKICBkZWZhdWx0X3VzZXI6CiAgICBuYW1lOiBveGlkZQ=="},"project_id":"473b39f9-8669-45b3-b3d7-793f7e10fc31","serialized_authn":{"kind":{"Authenticated":[{"actor":{"SiloUser":{"silo_id":"001de000-5110-4000-8000-000000000000","silo_user_id":"001de000-05e4-4000-8000-000000004007"}}},{"mapped_fleet_roles":{}}]}}},"which":15}}},{"SubsagaStart":{"params_node_name":"external_ip_params15","saga_name":"instance-create-external-ip15"}},{"Action":{"action_name":"instance_create.create_external_ip","label":"CreateExternalIp15","name":"external-ip-15"}},{"SubsagaEnd":{"name":"external_ip15"}},{"Constant":{"name":"external_ip_params16","value":{"instance_id":"d7bd2deb-ee27-4661-9e9a-524d734cd6ec","new_id":"f25005d7-f45c-4620-a7ed-94d2eda156e0","saga_params":{"boundary_switches":[],"create_params":{"anti_affinity_groups":[],"auto_restart_policy":null,"boot_disk":null,"description":"instance \\"utilization-test-instance\\"","disks":[],"external_ips":[],"hostname":"the-host","memory":1073741824,"name":"utilization-test-instance","ncpus":4,"network_interfaces":{"type":"default"},"ssh_public_keys":[],"start":true,"user_data":"I2Nsb3VkLWNvbmZpZwpzeXN0ZW1faW5mbzoKICBkZWZhdWx0X3VzZXI6CiAgICBuYW1lOiBveGlkZQ=="},"project_id":"473b39f9-8669-45b3-b3d7-793f7e10fc31","serialized_authn":{"kind":{"Authenticated":[{"actor":{"SiloUser":{"silo_id":"001de000-5110-4000-8000-000000000000","silo_user_id":"001de000-05e4-4000-8000-000000004007"}}},{"mapped_fleet_roles":{}}]}}},"which":16}}},{"SubsagaStart":{"params_node_name":"external_ip_params16","saga_name":"instance-create-external-ip16"}},{"Action":{"action_name":"instance_create.create_external_ip","label":"CreateExternalIp16","name":"external-ip-16"}},{"SubsagaEnd":{"name":"external_ip16"}},{"Constant":{"name":"external_ip_params17","value":{"instance_id":"d7bd2deb-ee27-4661-9e9a-524d734cd6ec","new_id":"7340d976-adcc-4ae0-b83b-0dd30bc87c58","saga_params":{"boundary_switches":[],"create_params":{"anti_affinity_groups":[],"auto_restart_policy":null,"boot_disk":null,"description":"instance \\"utilization-test-instance\\"","disks":[],"external_ips":[],"hostname":"the-host","memory":1073741824,"name":"utilization-test-instance","ncpus":4,"network_interfaces":{"type":"default"},"ssh_public_keys":[],"start":true,"user_data":"I2Nsb3VkLWNvbmZpZwpzeXN0ZW1faW5mbzoKICBkZWZhdWx0X3VzZXI6CiAgICBuYW1lOiBveGlkZQ=="},"project_id":"473b39f9-8669-45b3-b3d7-793f7e10fc31","serialized_authn":{"kind":{"Authenticated":[{"actor":{"SiloUser":{"silo_id":"001de000-5110-4000-8000-000000000000","silo_user_id":"001de000-05e4-4000-8000-000000004007"}}},{"mapped_fleet_roles":{}}]}}},"which":17}}},{"SubsagaStart":{"params_node_name":"external_ip_params17","saga_name":"instance-create-external-ip17"}},{"Action":{"action_name":"instance_create.create_external_ip","label":"CreateExternalIp17","name":"external-ip-17"}},{"SubsagaEnd":{"name":"external_ip17"}},{"Constant":{"name":"external_ip_params18","value":{"instance_id":"d7bd2deb-ee27-4661-9e9a-524d734cd6ec","new_id":"05e54cff-eed5-453a-98a1-297d8872176d","saga_params":{"boundary_switches":[],"create_params":{"anti_affinity_groups":[],"auto_restart_policy":null,"boot_disk":null,"description":"instance \\"utilization-test-instance\\"","disks":[],"external_ips":[],"hostname":"the-host","memory":1073741824,"name":"utilization-test-instance","ncpus":4,"network_interfaces":{"type":"default"},"ssh_public_keys":[],"start":true,"user_data":"I2Nsb3VkLWNvbmZpZwpzeXN0ZW1faW5mbzoKICBkZWZhdWx0X3VzZXI6CiAgICBuYW1lOiBveGlkZQ=="},"project_id":"473b39f9-8669-45b3-b3d7-793f7e10fc31","serialized_authn":{"kind":{"Authenticated":[{"actor":{"SiloUser":{"silo_id":"001de000-5110-4000-8000-000000000000","silo_user_id":"001de000-05e4-4000-8000-000000004007"}}},{"mapped_fleet_roles":{}}]}}},"which":18}}},{"SubsagaStart":{"params_node_name":"external_ip_params18","saga_name":"instance-create-external-ip18"}},{"Action":{"action_name":"instance_create.create_external_ip","label":"CreateExternalIp18","name":"external-ip-18"}},{"SubsagaEnd":{"name":"external_ip18"}},{"Constant":{"name":"external_ip_params19","value":{"instance_id":"d7bd2deb-ee27-4661-9e9a-524d734cd6ec","new_id":"fdb16759-a0d9-45ed-ac3f-ea7a5f13ab24","saga_params":{"boundary_switches":[],"create_params":{"anti_affinity_groups":[],"auto_restart_policy":null,"boot_disk":null,"description":"instance \\"utilization-test-instance\\"","disks":[],"external_ips":[],"hostname":"the-host","memory":1073741824,"name":"utilization-test-instance","ncpus":4,"network_interfaces":{"type":"default"},"ssh_public_keys":[],"start":true,"user_data":"I2Nsb3VkLWNvbmZpZwpzeXN0ZW1faW5mbzoKICBkZWZhdWx0X3VzZXI6CiAgICBuYW1lOiBveGlkZQ=="},"project_id":"473b39f9-8669-45b3-b3d7-793f7e10fc31","serialized_authn":{"kind":{"Authenticated":[{"actor":{"SiloUser":{"silo_id":"001de000-5110-4000-8000-000000000000","silo_user_id":"001de000-05e4-4000-8000-000000004007"}}},{"mapped_fleet_roles":{}}]}}},"which":19}}},{"SubsagaStart":{"params_node_name":"external_ip_params19","saga_name":"instance-create-external-ip19"}},{"Action":{"action_name":"instance_create.create_external_ip","label":"CreateExternalIp19","name":"external-ip-19"}},{"SubsagaEnd":{"name":"external_ip19"}},{"Constant":{"name":"external_ip_params20","value":{"instance_id":"d7bd2deb-ee27-4661-9e9a-524d734cd6ec","new_id":"b19a9736-163c-49c2-8115-ee28cc17761a","saga_params":{"boundary_switches":[],"create_params":{"anti_affinity_groups":[],"auto_restart_policy":null,"boot_disk":null,"description":"instance \\"utilization-test-instance\\"","disks":[],"external_ips":[],"hostname":"the-host","memory":1073741824,"name":"utilization-test-instance","ncpus":4,"network_interfaces":{"type":"default"},"ssh_public_keys":[],"start":true,"user_data":"I2Nsb3VkLWNvbmZpZwpzeXN0ZW1faW5mbzoKICBkZWZhdWx0X3VzZXI6CiAgICBuYW1lOiBveGlkZQ=="},"project_id":"473b39f9-8669-45b3-b3d7-793f7e10fc31","serialized_authn":{"kind":{"Authenticated":[{"actor":{"SiloUser":{"silo_id":"001de000-5110-4000-8000-000000000000","silo_user_id":"001de000-05e4-4000-8000-000000004007"}}},{"mapped_fleet_roles":{}}]}}},"which":20}}},{"SubsagaStart":{"params_node_name":"external_ip_params20","saga_name":"instance-create-external-ip20"}},{"Action":{"action_name":"instance_create.create_external_ip","label":"CreateExternalIp20","name":"external-ip-20"}},{"SubsagaEnd":{"name":"external_ip20"}},{"Constant":{"name":"external_ip_params21","value":{"instance_id":"d7bd2deb-ee27-4661-9e9a-524d734cd6ec","new_id":"3e95bbf0-31cc-42db-9c13-91a80858663c","saga_params":{"boundary_switches":[],"create_params":{"anti_affinity_groups":[],"auto_restart_policy":null,"boot_disk":null,"description":"instance \\"utilization-test-instance\\"","disks":[],"external_ips":[],"hostname":"the-host","memory":1073741824,"name":"utilization-test-instance","ncpus":4,"network_interfaces":{"type":"default"},"ssh_public_keys":[],"start":true,"user_data":"I2Nsb3VkLWNvbmZpZwpzeXN0ZW1faW5mbzoKICBkZWZhdWx0X3VzZXI6CiAgICBuYW1lOiBveGlkZQ=="},"project_id":"473b39f9-8669-45b3-b3d7-793f7e10fc31","serialized_authn":{"kind":{"Authenticated":[{"actor":{"SiloUser":{"silo_id":"001de000-5110-4000-8000-000000000000","silo_user_id":"001de000-05e4-4000-8000-000000004007"}}},{"mapped_fleet_roles":{}}]}}},"which":21}}},{"SubsagaStart":{"params_node_name":"external_ip_params21","saga_name":"instance-create-external-ip21"}},{"Action":{"action_name":"instance_create.create_external_ip","label":"CreateExternalIp21","name":"external-ip-21"}},{"SubsagaEnd":{"name":"external_ip21"}},{"Constant":{"name":"external_ip_params22","value":{"instance_id":"d7bd2deb-ee27-4661-9e9a-524d734cd6ec","new_id":"a4cb59af-13e0-43d4-9e6a-cb804af95f94","saga_params":{"boundary_switches":[],"create_params":{"anti_affinity_groups":[],"auto_restart_policy":null,"boot_disk":null,"description":"instance \\"utilization-test-instance\\"","disks":[],"external_ips":[],"hostname":"the-host","memory":1073741824,"name":"utilization-test-instance","ncpus":4,"network_interfaces":{"type":"default"},"ssh_public_keys":[],"start":true,"user_data":"I2Nsb3VkLWNvbmZpZwpzeXN0ZW1faW5mbzoKICBkZWZhdWx0X3VzZXI6CiAgICBuYW1lOiBveGlkZQ=="},"project_id":"473b39f9-8669-45b3-b3d7-793f7e10fc31","serialized_authn":{"kind":{"Authenticated":[{"actor":{"SiloUser":{"silo_id":"001de000-5110-4000-8000-000000000000","silo_user_id":"001de000-05e4-4000-8000-000000004007"}}},{"mapped_fleet_roles":{}}]}}},"which":22}}},{"SubsagaStart":{"params_node_name":"external_ip_params22","saga_name":"instance-create-external-ip22"}},{"Action":{"action_name":"instance_create.create_external_ip","label":"CreateExternalIp22","name":"external-ip-22"}},{"SubsagaEnd":{"name":"external_ip22"}},{"Constant":{"name":"external_ip_params23","value":{"instance_id":"d7bd2deb-ee27-4661-9e9a-524d734cd6ec","new_id":"f4bf029b-6ff1-4269-a3aa-78663047295b","saga_params":{"boundary_switches":[],"create_params":{"anti_affinity_groups":[],"auto_restart_policy":null,"boot_disk":null,"description":"instance \\"utilization-test-instance\\"","disks":[],"external_ips":[],"hostname":"the-host","memory":1073741824,"name":"utilization-test-instance","ncpus":4,"network_interfaces":{"type":"default"},"ssh_public_keys":[],"start":true,"user_data":"I2Nsb3VkLWNvbmZpZwpzeXN0ZW1faW5mbzoKICBkZWZhdWx0X3VzZXI6CiAgICBuYW1lOiBveGlkZQ=="},"project_id":"473b39f9-8669-45b3-b3d7-793f7e10fc31","serialized_authn":{"kind":{"Authenticated":[{"actor":{"SiloUser":{"silo_id":"001de000-5110-4000-8000-000000000000","silo_user_id":"001de000-05e4-4000-8000-000000004007"}}},{"mapped_fleet_roles":{}}]}}},"which":23}}},{"SubsagaStart":{"params_node_name":"external_ip_params23","saga_name":"instance-create-external-ip23"}},{"Action":{"action_name":"instance_create.create_external_ip","label":"CreateExternalIp23","name":"external-ip-23"}},{"SubsagaEnd":{"name":"external_ip23"}},{"Constant":{"name":"external_ip_params24","value":{"instance_id":"d7bd2deb-ee27-4661-9e9a-524d734cd6ec","new_id":"34abc739-27ab-4357-820a-4bd83d55c78b","saga_params":{"boundary_switches":[],"create_params":{"anti_affinity_groups":[],"auto_restart_policy":null,"boot_disk":null,"description":"instance \\"utilization-test-instance\\"","disks":[],"external_ips":[],"hostname":"the-host","memory":1073741824,"name":"utilization-test-instance","ncpus":4,"network_interfaces":{"type":"default"},"ssh_public_keys":[],"start":true,"user_data":"I2Nsb3VkLWNvbmZpZwpzeXN0ZW1faW5mbzoKICBkZWZhdWx0X3VzZXI6CiAgICBuYW1lOiBveGlkZQ=="},"project_id":"473b39f9-8669-45b3-b3d7-793f7e10fc31","serialized_authn":{"kind":{"Authenticated":[{"actor":{"SiloUser":{"silo_id":"001de000-5110-4000-8000-000000000000","silo_user_id":"001de000-05e4-4000-8000-000000004007"}}},{"mapped_fleet_roles":{}}]}}},"which":24}}},{"SubsagaStart":{"params_node_name":"external_ip_params24","saga_name":"instance-create-external-ip24"}},{"Action":{"action_name":"instance_create.create_external_ip","label":"CreateExternalIp24","name":"external-ip-24"}},{"SubsagaEnd":{"name":"external_ip24"}},{"Constant":{"name":"external_ip_params25","value":{"instance_id":"d7bd2deb-ee27-4661-9e9a-524d734cd6ec","new_id":"c4c3995e-045b-4152-ad96-930a44eb1c91","saga_params":{"boundary_switches":[],"create_params":{"anti_affinity_groups":[],"auto_restart_policy":null,"boot_disk":null,"description":"instance \\"utilization-test-instance\\"","disks":[],"external_ips":[],"hostname":"the-host","memory":1073741824,"name":"utilization-test-instance","ncpus":4,"network_interfaces":{"type":"default"},"ssh_public_keys":[],"start":true,"user_data":"I2Nsb3VkLWNvbmZpZwpzeXN0ZW1faW5mbzoKICBkZWZhdWx0X3VzZXI6CiAgICBuYW1lOiBveGlkZQ=="},"project_id":"473b39f9-8669-45b3-b3d7-793f7e10fc31","serialized_authn":{"kind":{"Authenticated":[{"actor":{"SiloUser":{"silo_id":"001de000-5110-4000-8000-000000000000","silo_user_id":"001de000-05e4-4000-8000-000000004007"}}},{"mapped_fleet_roles":{}}]}}},"which":25}}},{"SubsagaStart":{"params_node_name":"external_ip_params25","saga_name":"instance-create-external-ip25"}},{"Action":{"action_name":"instance_create.create_external_ip","label":"CreateExternalIp25","name":"external-ip-25"}},{"SubsagaEnd":{"name":"external_ip25"}},{"Constant":{"name":"external_ip_params26","value":{"instance_id":"d7bd2deb-ee27-4661-9e9a-524d734cd6ec","new_id":"aa70062c-595c-4b83-8d78-1efc9c2f5978","saga_params":{"boundary_switches":[],"create_params":{"anti_affinity_groups":[],"auto_restart_policy":null,"boot_disk":null,"description":"instance \\"utilization-test-instance\\"","disks":[],"external_ips":[],"hostname":"the-host","memory":1073741824,"name":"utilization-test-instance","ncpus":4,"network_interfaces":{"type":"default"},"ssh_public_keys":[],"start":true,"user_data":"I2Nsb3VkLWNvbmZpZwpzeXN0ZW1faW5mbzoKICBkZWZhdWx0X3VzZXI6CiAgICBuYW1lOiBveGlkZQ=="},"project_id":"473b39f9-8669-45b3-b3d7-793f7e10fc31","serialized_authn":{"kind":{"Authenticated":[{"actor":{"SiloUser":{"silo_id":"001de000-5110-4000-8000-000000000000","silo_user_id":"001de000-05e4-4000-8000-000000004007"}}},{"mapped_fleet_roles":{}}]}}},"which":26}}},{"SubsagaStart":{"params_node_name":"external_ip_params26","saga_name":"instance-create-external-ip26"}},{"Action":{"action_name":"instance_create.create_external_ip","label":"CreateExternalIp26","name":"external-ip-26"}},{"SubsagaEnd":{"name":"external_ip26"}},{"Constant":{"name":"external_ip_params27","value":{"instance_id":"d7bd2deb-ee27-4661-9e9a-524d734cd6ec","new_id":"25bbbf93-a75b-4a07-b175-ad5ff6de72e6","saga_params":{"boundary_switches":[],"create_params":{"anti_affinity_groups":[],"auto_restart_policy":null,"boot_disk":null,"description":"instance \\"utilization-test-instance\\"","disks":[],"external_ips":[],"hostname":"the-host","memory":1073741824,"name":"utilization-test-instance","ncpus":4,"network_interfaces":{"type":"default"},"ssh_public_keys":[],"start":true,"user_data":"I2Nsb3VkLWNvbmZpZwpzeXN0ZW1faW5mbzoKICBkZWZhdWx0X3VzZXI6CiAgICBuYW1lOiBveGlkZQ=="},"project_id":"473b39f9-8669-45b3-b3d7-793f7e10fc31","serialized_authn":{"kind":{"Authenticated":[{"actor":{"SiloUser":{"silo_id":"001de000-5110-4000-8000-000000000000","silo_user_id":"001de000-05e4-4000-8000-000000004007"}}},{"mapped_fleet_roles":{}}]}}},"which":27}}},{"SubsagaStart":{"params_node_name":"external_ip_params27","saga_name":"instance-create-external-ip27"}},{"Action":{"action_name":"instance_create.create_external_ip","label":"CreateExternalIp27","name":"external-ip-27"}},{"SubsagaEnd":{"name":"external_ip27"}},{"Constant":{"name":"external_ip_params28","value":{"instance_id":"d7bd2deb-ee27-4661-9e9a-524d734cd6ec","new_id":"6047d897-d5be-45b8-93ff-0710079dfeae","saga_params":{"boundary_switches":[],"create_params":{"anti_affinity_groups":[],"auto_restart_policy":null,"boot_disk":null,"description":"instance \\"utilization-test-instance\\"","disks":[],"external_ips":[],"hostname":"the-host","memory":1073741824,"name":"utilization-test-instance","ncpus":4,"network_interfaces":{"type":"default"},"ssh_public_keys":[],"start":true,"user_data":"I2Nsb3VkLWNvbmZpZwpzeXN0ZW1faW5mbzoKICBkZWZhdWx0X3VzZXI6CiAgICBuYW1lOiBveGlkZQ=="},"project_id":"473b39f9-8669-45b3-b3d7-793f7e10fc31","serialized_authn":{"kind":{"Authenticated":[{"actor":{"SiloUser":{"silo_id":"001de000-5110-4000-8000-000000000000","silo_user_id":"001de000-05e4-4000-8000-000000004007"}}},{"mapped_fleet_roles":{}}]}}},"which":28}}},{"SubsagaStart":{"params_node_name":"external_ip_params28","saga_name":"instance-create-external-ip28"}},{"Action":{"action_name":"instance_create.create_external_ip","label":"CreateExternalIp28","name":"external-ip-28"}},{"SubsagaEnd":{"name":"external_ip28"}},{"Constant":{"name":"external_ip_params29","value":{"instance_id":"d7bd2deb-ee27-4661-9e9a-524d734cd6ec","new_id":"fe58941b-b766-487e-812e-f5e79533a730","saga_params":{"boundary_switches":[],"create_params":{"anti_affinity_groups":[],"auto_restart_policy":null,"boot_disk":null,"description":"instance \\"utilization-test-instance\\"","disks":[],"external_ips":[],"hostname":"the-host","memory":1073741824,"name":"utilization-test-instance","ncpus":4,"network_interfaces":{"type":"default"},"ssh_public_keys":[],"start":true,"user_data":"I2Nsb3VkLWNvbmZpZwpzeXN0ZW1faW5mbzoKICBkZWZhdWx0X3VzZXI6CiAgICBuYW1lOiBveGlkZQ=="},"project_id":"473b39f9-8669-45b3-b3d7-793f7e10fc31","serialized_authn":{"kind":{"Authenticated":[{"actor":{"SiloUser":{"silo_id":"001de000-5110-4000-8000-000000000000","silo_user_id":"001de000-05e4-4000-8000-000000004007"}}},{"mapped_fleet_roles":{}}]}}},"which":29}}},{"SubsagaStart":{"params_node_name":"external_ip_params29","saga_name":"instance-create-external-ip29"}},{"Action":{"action_name":"instance_create.create_external_ip","label":"CreateExternalIp29","name":"external-ip-29"}},{"SubsagaEnd":{"name":"external_ip29"}},{"Constant":{"name":"external_ip_params30","value":{"instance_id":"d7bd2deb-ee27-4661-9e9a-524d734cd6ec","new_id":"d4aeb710-e637-4ddd-b35b-dd02d5495d12","saga_params":{"boundary_switches":[],"create_params":{"anti_affinity_groups":[],"auto_restart_policy":null,"boot_disk":null,"description":"instance \\"utilization-test-instance\\"","disks":[],"external_ips":[],"hostname":"the-host","memory":1073741824,"name":"utilization-test-instance","ncpus":4,"network_interfaces":{"type":"default"},"ssh_public_keys":[],"start":true,"user_data":"I2Nsb3VkLWNvbmZpZwpzeXN0ZW1faW5mbzoKICBkZWZhdWx0X3VzZXI6CiAgICBuYW1lOiBveGlkZQ=="},"project_id":"473b39f9-8669-45b3-b3d7-793f7e10fc31","serialized_authn":{"kind":{"Authenticated":[{"actor":{"SiloUser":{"silo_id":"001de000-5110-4000-8000-000000000000","silo_user_id":"001de000-05e4-4000-8000-000000004007"}}},{"mapped_fleet_roles":{}}]}}},"which":30}}},{"SubsagaStart":{"params_node_name":"external_ip_params30","saga_name":"instance-create-external-ip30"}},{"Action":{"action_name":"instance_create.create_external_ip","label":"CreateExternalIp30","name":"external-ip-30"}},{"SubsagaEnd":{"name":"external_ip30"}},{"Constant":{"name":"external_ip_params31","value":{"instance_id":"d7bd2deb-ee27-4661-9e9a-524d734cd6ec","new_id":"28acecea-be1f-451f-ac0d-066b7436b4d2","saga_params":{"boundary_switches":[],"create_params":{"anti_affinity_groups":[],"auto_restart_policy":null,"boot_disk":null,"description":"instance \\"utilization-test-instance\\"","disks":[],"external_ips":[],"hostname":"the-host","memory":1073741824,"name":"utilization-test-instance","ncpus":4,"network_interfaces":{"type":"default"},"ssh_public_keys":[],"start":true,"user_data":"I2Nsb3VkLWNvbmZpZwpzeXN0ZW1faW5mbzoKICBkZWZhdWx0X3VzZXI6CiAgICBuYW1lOiBveGlkZQ=="},"project_id":"473b39f9-8669-45b3-b3d7-793f7e10fc31","serialized_authn":{"kind":{"Authenticated":[{"actor":{"SiloUser":{"silo_id":"001de000-5110-4000-8000-000000000000","silo_user_id":"001de000-05e4-4000-8000-000000004007"}}},{"mapped_fleet_roles":{}}]}}},"which":31}}},{"SubsagaStart":{"params_node_name":"external_ip_params31","saga_name":"instance-create-external-ip31"}},{"Action":{"action_name":"instance_create.create_external_ip","label":"CreateExternalIp31","name":"external-ip-31"}},{"SubsagaEnd":{"name":"external_ip31"}},{"Action":{"action_name":"instance_create.set_boot_disk","label":"SetBootDisk","name":"set_boot_disk"}},{"Action":{"action_name":"instance_create.move_to_stopped","label":"MoveToStopped","name":"stopped_instance"}},{"Start":{"params":{"boundary_switches":[],"create_params":{"anti_affinity_groups":[],"auto_restart_policy":null,"boot_disk":null,"description":"instance \\"utilization-test-instance\\"","disks":[],"external_ips":[],"hostname":"the-host","memory":1073741824,"name":"utilization-test-instance","ncpus":4,"network_interfaces":{"type":"default"},"ssh_public_keys":[],"start":true,"user_data":"I2Nsb3VkLWNvbmZpZwpzeXN0ZW1faW5mbzoKICBkZWZhdWx0X3VzZXI6CiAgICBuYW1lOiBveGlkZQ=="},"project_id":"473b39f9-8669-45b3-b3d7-793f7e10fc31","serialized_authn":{"kind":{"Authenticated":[{"actor":{"SiloUser":{"silo_id":"001de000-5110-4000-8000-000000000000","silo_user_id":"001de000-05e4-4000-8000-000000004007"}}},{"mapped_fleet_roles":{}}]}}}}},"End"]},"saga_name":"instance-create","start_node":167} saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c saga_name = instance-create sec_id = ac55d8d9-a298-4534-898b-beee1c9e4e52
40502025-04-30T06:43:53.956ZINFOtest_utilization (SecStore): creating saga saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c saga_name = instance-create
40512025-04-30T06:43:53.960ZINFOtest_utilization (oximeter-agent): refreshing list of producers from Nexus collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 file = oximeter/collector/src/agent.rs:562
40522025-04-30T06:43:53.960ZDEBGtest_utilization (oximeter): client request body = None method = GET uri = http://127.0.0.1:32956/metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?sort_by=id_ascending
40532025-04-30T06:43:53.961ZTRACtest_utilization (dropshot_internal): incoming request local_addr = 127.0.0.1:32956 method = GET remote_addr = 127.0.0.1:55797 req_id = 1bb2e01a-e849-4860-847c-fc2449f1179b uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?sort_by=id_ascending
40542025-04-30T06:43:53.961ZTRACtest_utilization (dropshot_internal): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = 127.0.0.1:32956 method = GET remote_addr = 127.0.0.1:55797 req_id = 1bb2e01a-e849-4860-847c-fc2449f1179b resource = Database uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?sort_by=id_ascending
40552025-04-30T06:43:53.961ZDEBGtest_utilization (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = 127.0.0.1:32956 method = GET remote_addr = 127.0.0.1:55797 req_id = 1bb2e01a-e849-4860-847c-fc2449f1179b roles = RoleSet { roles: {} } uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?sort_by=id_ascending
40562025-04-30T06:43:53.964ZDEBGtest_utilization (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = 127.0.0.1:32956 method = GET remote_addr = 127.0.0.1:55797 req_id = 1bb2e01a-e849-4860-847c-fc2449f1179b resource = Database result = Ok(()) uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?sort_by=id_ascending
40572025-04-30T06:43:53.969ZINFOtest_utilization (dropshot_internal): request completed latency_us = 8025 local_addr = 127.0.0.1:32956 method = GET remote_addr = 127.0.0.1:55797 req_id = 1bb2e01a-e849-4860-847c-fc2449f1179b response_code = 200 uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?sort_by=id_ascending
40582025-04-30T06:43:53.969ZDEBGtest_utilization (oximeter): client response result = Ok(Response { url: "http://127.0.0.1:32956/metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?sort_by=id_ascending", status: 200, headers: {"content-type": "application/json", "x-request-id": "1bb2e01a-e849-4860-847c-fc2449f1179b", "content-length": "405", "date": "Wed, 30 Apr 2025 06:43:53 GMT"} })
40592025-04-30T06:43:53.970ZDEBGtest_utilization (oximeter): client request body = None method = GET uri = http://127.0.0.1:32956/metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJhYzU1ZDhkOS1hMjk4LTQ1MzQtODk4Yi1iZWVlMWM5ZTRlNTIifX0%3D
40602025-04-30T06:43:53.970ZTRACtest_utilization (dropshot_internal): incoming request local_addr = 127.0.0.1:32956 method = GET remote_addr = 127.0.0.1:55797 req_id = 6ff313a3-1012-4d20-a578-bc9ea1cdee97 uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJhYzU1ZDhkOS1hMjk4LTQ1MzQtODk4Yi1iZWVlMWM5ZTRlNTIifX0%3D
40612025-04-30T06:43:53.972ZTRACtest_utilization (dropshot_internal): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = 127.0.0.1:32956 method = GET remote_addr = 127.0.0.1:55797 req_id = 6ff313a3-1012-4d20-a578-bc9ea1cdee97 resource = Database uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJhYzU1ZDhkOS1hMjk4LTQ1MzQtODk4Yi1iZWVlMWM5ZTRlNTIifX0%3D
40622025-04-30T06:43:53.972ZDEBGtest_utilization (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = 127.0.0.1:32956 method = GET remote_addr = 127.0.0.1:55797 req_id = 6ff313a3-1012-4d20-a578-bc9ea1cdee97 roles = RoleSet { roles: {} } uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJhYzU1ZDhkOS1hMjk4LTQ1MzQtODk4Yi1iZWVlMWM5ZTRlNTIifX0%3D
40632025-04-30T06:43:53.973ZDEBGtest_utilization (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = 127.0.0.1:32956 method = GET remote_addr = 127.0.0.1:55797 req_id = 6ff313a3-1012-4d20-a578-bc9ea1cdee97 resource = Database result = Ok(()) uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJhYzU1ZDhkOS1hMjk4LTQ1MzQtODk4Yi1iZWVlMWM5ZTRlNTIifX0%3D
40642025-04-30T06:43:53.979ZINFOtest_utilization (SagaExecutor): starting saga saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c saga_name = instance-create
40652025-04-30T06:43:53.979ZINFOtest_utilization (SEC): saga start saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c saga_name = instance-create sec_id = ac55d8d9-a298-4534-898b-beee1c9e4e52
40662025-04-30T06:43:53.986ZDEBGtest_utilization (SEC): saga log event new_state = N167 started sec_id = ac55d8d9-a298-4534-898b-beee1c9e4e52
40672025-04-30T06:43:53.986ZDEBGtest_utilization (SecStore): recording saga event event_type = Started node_id = 167 saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c
40682025-04-30T06:43:53.991ZINFOtest_utilization (dropshot_internal): request completed latency_us = 20878 local_addr = 127.0.0.1:32956 method = GET remote_addr = 127.0.0.1:55797 req_id = 6ff313a3-1012-4d20-a578-bc9ea1cdee97 response_code = 200 uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJhYzU1ZDhkOS1hMjk4LTQ1MzQtODk4Yi1iZWVlMWM5ZTRlNTIifX0%3D
40692025-04-30T06:43:53.992ZDEBGtest_utilization (SEC): saga log event new_state = N167 succeeded sec_id = ac55d8d9-a298-4534-898b-beee1c9e4e52
40702025-04-30T06:43:53.992ZDEBGtest_utilization (SecStore): recording saga event event_type = Succeeded(Null) node_id = 167 saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c
40712025-04-30T06:43:53.994ZDEBGtest_utilization (oximeter): client response result = Ok(Response { url: "http://127.0.0.1:32956/metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJhYzU1ZDhkOS1hMjk4LTQ1MzQtODk4Yi1iZWVlMWM5ZTRlNTIifX0%3D", status: 200, headers: {"content-type": "application/json", "x-request-id": "6ff313a3-1012-4d20-a578-bc9ea1cdee97", "content-length": "29", "date": "Wed, 30 Apr 2025 06:43:53 GMT"} })
40722025-04-30T06:43:53.995ZDEBGtest_utilization (oximeter-agent): registered new metric producer address = [::1]:54359 collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
40732025-04-30T06:43:54.106ZTRACtest_utilization (collection-task): top of inner collection loop, waiting for next request collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
40742025-04-30T06:43:54.138ZDEBGtest_utilization (oximeter-agent): registered new metric producer address = 127.0.0.1:57688 collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = ac55d8d9-a298-4534-898b-beee1c9e4e52
40752025-04-30T06:43:54.222ZTRACtest_utilization (collection-task): top of inner collection loop, waiting for next request collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = ac55d8d9-a298-4534-898b-beee1c9e4e52
40762025-04-30T06:43:54.223ZDEBGtest_utilization (SEC): saga log event new_state = N000 started sec_id = ac55d8d9-a298-4534-898b-beee1c9e4e52
40772025-04-30T06:43:54.223ZDEBGtest_utilization (SecStore): recording saga event event_type = Started node_id = 0 saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c
40782025-04-30T06:43:54.224ZINFOtest_utilization (oximeter-agent): refreshed list of producers from Nexus collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 file = oximeter/collector/src/agent.rs:637 n_current_tasks = 2 n_pruned_tasks = 0
40792025-04-30T06:43:54.230ZDEBGtest_utilization (SEC): saga log event new_state = N000 succeeded sec_id = ac55d8d9-a298-4534-898b-beee1c9e4e52
40802025-04-30T06:43:54.230ZDEBGtest_utilization (SecStore): recording saga event event_type = Succeeded(String("d7bd2deb-ee27-4661-9e9a-524d734cd6ec")) node_id = 0 saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c
40812025-04-30T06:43:54.237ZDEBGtest_utilization (SEC): saga log event new_state = N001 started sec_id = ac55d8d9-a298-4534-898b-beee1c9e4e52
40822025-04-30T06:43:54.237ZDEBGtest_utilization (SecStore): recording saga event event_type = Started node_id = 1 saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c
40832025-04-30T06:43:54.249ZTRACtest_utilization (SagaExecutor): authorize begin action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true resource = Database saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c saga_name = instance-create saga_node = CreateInstanceRecord
40842025-04-30T06:43:54.249ZDEBGtest_utilization (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true roles = RoleSet { roles: {} } saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c saga_name = instance-create saga_node = CreateInstanceRecord
40852025-04-30T06:43:54.250ZDEBGtest_utilization (SagaExecutor): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true resource = Database result = Ok(()) saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c saga_name = instance-create saga_node = CreateInstanceRecord
40862025-04-30T06:43:54.270ZTRACtest_utilization (SagaExecutor): authorize begin action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true resource = Database saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c saga_name = instance-create saga_node = CreateInstanceRecord
40872025-04-30T06:43:54.270ZDEBGtest_utilization (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true roles = RoleSet { roles: {} } saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c saga_name = instance-create saga_node = CreateInstanceRecord
40882025-04-30T06:43:54.270ZDEBGtest_utilization (SagaExecutor): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true resource = Database result = Ok(()) saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c saga_name = instance-create saga_node = CreateInstanceRecord
40892025-04-30T06:43:54.276ZTRACtest_utilization (SagaExecutor): authorize begin action = CreateChild actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true resource = Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000000, lookup_type: ById(001de000-5110-4000-8000-000000000000) }, key: 473b39f9-8669-45b3-b3d7-793f7e10fc31, lookup_type: ById(473b39f9-8669-45b3-b3d7-793f7e10fc31) } saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c saga_name = instance-create saga_node = CreateInstanceRecord
40902025-04-30T06:43:54.276ZTRACtest_utilization (SagaExecutor): loading roles actor = Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. } actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true resource_id = 473b39f9-8669-45b3-b3d7-793f7e10fc31 resource_type = Project saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c saga_name = instance-create saga_node = CreateInstanceRecord
40912025-04-30T06:43:54.276ZTRACtest_utilization (SagaExecutor): authorize begin action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true resource = Database saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c saga_name = instance-create saga_node = CreateInstanceRecord
40922025-04-30T06:43:54.276ZDEBGtest_utilization (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true roles = RoleSet { roles: {} } saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c saga_name = instance-create saga_node = CreateInstanceRecord
40932025-04-30T06:43:54.277ZDEBGtest_utilization (SagaExecutor): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true resource = Database result = Ok(()) saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c saga_name = instance-create saga_node = CreateInstanceRecord
40942025-04-30T06:43:54.298ZTRACtest_utilization (SagaExecutor): loading roles actor = Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. } actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true resource_id = 001de000-5110-4000-8000-000000000000 resource_type = Silo saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c saga_name = instance-create saga_node = CreateInstanceRecord
40952025-04-30T06:43:54.298ZTRACtest_utilization (SagaExecutor): authorize begin action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true resource = Database saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c saga_name = instance-create saga_node = CreateInstanceRecord
40962025-04-30T06:43:54.298ZDEBGtest_utilization (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true roles = RoleSet { roles: {} } saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c saga_name = instance-create saga_node = CreateInstanceRecord
40972025-04-30T06:43:54.300ZDEBGtest_utilization (SagaExecutor): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true resource = Database result = Ok(()) saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c saga_name = instance-create saga_node = CreateInstanceRecord
40982025-04-30T06:43:54.308ZTRACtest_utilization (SagaExecutor): loading roles actor = Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. } actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c saga_name = instance-create saga_node = CreateInstanceRecord
40992025-04-30T06:43:54.308ZTRACtest_utilization (SagaExecutor): authorize begin action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true resource = Database saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c saga_name = instance-create saga_node = CreateInstanceRecord
41002025-04-30T06:43:54.310ZDEBGtest_utilization (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true roles = RoleSet { roles: {} } saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c saga_name = instance-create saga_node = CreateInstanceRecord
41012025-04-30T06:43:54.323ZDEBGtest_utilization (SagaExecutor): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true resource = Database result = Ok(()) saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c saga_name = instance-create saga_node = CreateInstanceRecord
41022025-04-30T06:43:54.331ZDEBGtest_utilization (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin"), (Silo, 001de000-5110-4000-8000-000000000000, "admin")} } saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c saga_name = instance-create saga_node = CreateInstanceRecord
41032025-04-30T06:43:54.339ZDEBGtest_utilization (SagaExecutor): authorize result action = CreateChild actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true resource = Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000000, lookup_type: ById(001de000-5110-4000-8000-000000000000) }, key: 473b39f9-8669-45b3-b3d7-793f7e10fc31, lookup_type: ById(473b39f9-8669-45b3-b3d7-793f7e10fc31) } result = Ok(()) saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c saga_name = instance-create saga_node = CreateInstanceRecord
41042025-04-30T06:43:54.339ZTRACtest_utilization (SagaExecutor): authorize begin action = CreateChild actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true resource = Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000000, lookup_type: ById(001de000-5110-4000-8000-000000000000) }, key: 473b39f9-8669-45b3-b3d7-793f7e10fc31, lookup_type: ById(473b39f9-8669-45b3-b3d7-793f7e10fc31) } saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c saga_name = instance-create saga_node = CreateInstanceRecord
41052025-04-30T06:43:54.340ZTRACtest_utilization (SagaExecutor): loading roles actor = Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. } actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true resource_id = 473b39f9-8669-45b3-b3d7-793f7e10fc31 resource_type = Project saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c saga_name = instance-create saga_node = CreateInstanceRecord
41062025-04-30T06:43:54.340ZTRACtest_utilization (SagaExecutor): authorize begin action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true resource = Database saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c saga_name = instance-create saga_node = CreateInstanceRecord
41072025-04-30T06:43:54.340ZDEBGtest_utilization (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true roles = RoleSet { roles: {} } saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c saga_name = instance-create saga_node = CreateInstanceRecord
41082025-04-30T06:43:54.348ZDEBGtest_utilization (SagaExecutor): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true resource = Database result = Ok(()) saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c saga_name = instance-create saga_node = CreateInstanceRecord
41092025-04-30T06:43:54.361ZTRACtest_utilization (results-sink): batch interval expired, but no samples to insert collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
41102025-04-30T06:43:54.361ZTRACtest_utilization (results-sink-cluster): batch interval expired, but no samples to insert collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
41112025-04-30T06:43:54.368ZTRACtest_utilization (SagaExecutor): loading roles actor = Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. } actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true resource_id = 001de000-5110-4000-8000-000000000000 resource_type = Silo saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c saga_name = instance-create saga_node = CreateInstanceRecord
41122025-04-30T06:43:54.368ZTRACtest_utilization (SagaExecutor): authorize begin action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true resource = Database saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c saga_name = instance-create saga_node = CreateInstanceRecord
41132025-04-30T06:43:54.368ZDEBGtest_utilization (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true roles = RoleSet { roles: {} } saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c saga_name = instance-create saga_node = CreateInstanceRecord
41142025-04-30T06:43:54.368ZDEBGtest_utilization (SagaExecutor): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true resource = Database result = Ok(()) saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c saga_name = instance-create saga_node = CreateInstanceRecord
41152025-04-30T06:43:54.379ZTRACtest_utilization (SagaExecutor): loading roles actor = Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. } actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c saga_name = instance-create saga_node = CreateInstanceRecord
41162025-04-30T06:43:54.379ZTRACtest_utilization (SagaExecutor): authorize begin action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true resource = Database saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c saga_name = instance-create saga_node = CreateInstanceRecord
41172025-04-30T06:43:54.379ZDEBGtest_utilization (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true roles = RoleSet { roles: {} } saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c saga_name = instance-create saga_node = CreateInstanceRecord
41182025-04-30T06:43:54.383ZDEBGtest_utilization (SagaExecutor): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true resource = Database result = Ok(()) saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c saga_name = instance-create saga_node = CreateInstanceRecord
41192025-04-30T06:43:54.387ZDEBGtest_utilization (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin"), (Silo, 001de000-5110-4000-8000-000000000000, "admin")} } saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c saga_name = instance-create saga_node = CreateInstanceRecord
41202025-04-30T06:43:54.396ZDEBGtest_utilization (SagaExecutor): authorize result action = CreateChild actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true resource = Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000000, lookup_type: ById(001de000-5110-4000-8000-000000000000) }, key: 473b39f9-8669-45b3-b3d7-793f7e10fc31, lookup_type: ById(473b39f9-8669-45b3-b3d7-793f7e10fc31) } result = Ok(()) saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c saga_name = instance-create saga_node = CreateInstanceRecord
41212025-04-30T06:43:54.409ZTRACtest_utilization (SagaExecutor): authorize begin action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true resource = Database saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c saga_name = instance-create saga_node = CreateInstanceRecord
41222025-04-30T06:43:54.409ZDEBGtest_utilization (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true roles = RoleSet { roles: {} } saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c saga_name = instance-create saga_node = CreateInstanceRecord
41232025-04-30T06:43:54.410ZDEBGtest_utilization (SagaExecutor): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true resource = Database result = Ok(()) saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c saga_name = instance-create saga_node = CreateInstanceRecord
41242025-04-30T06:43:54.506ZDEBGtest_utilization (SEC): saga log event new_state = N001 succeeded sec_id = ac55d8d9-a298-4534-898b-beee1c9e4e52
41252025-04-30T06:43:54.506ZDEBGtest_utilization (SecStore): recording saga event event_type = Succeeded(Object {"auto_restart": Object {"cooldown": Null, "policy": Null}, "boot_disk_id": Null, "hostname": String("the-host"), "identity": Object {"description": String("instance \\"utilization-test-instance\\""), "id": String("d7bd2deb-ee27-4661-9e9a-524d734cd6ec"), "name": String("utilization-test-instance"), "time_created": String("2025-04-30T06:43:54.249576Z"), "time_deleted": Null, "time_modified": String("2025-04-30T06:43:54.249576Z")}, "memory": Number(1073741824), "ncpus": Number(4), "project_id": String("473b39f9-8669-45b3-b3d7-793f7e10fc31"), "runtime_state": Object {"dst_propolis_id": Null, "gen": Number(1), "migration_id": Null, "nexus_state": String("Creating"), "propolis_id": Null, "time_last_auto_restarted": Null, "time_updated": String("2025-04-30T06:43:54.249576Z")}, "updater_gen": Number(1), "updater_id": Null, "user_data": Array [Number(35), Number(99), Number(108), Number(111), Number(117), Number(100), Number(45), Number(99), Number(111), Number(110), Number(102), Number(105), Number(103), Number(10), Number(115), Number(121), Number(115), Number(116), Number(101), Number(109), Number(95), Number(105), Number(110), Number(102), Number(111), Number(58), Number(10), Number(32), Number(32), Number(100), Number(101), Number(102), Number(97), Number(117), Number(108), Number(116), Number(95), Number(117), Number(115), Number(101), Number(114), Number(58), Number(10), Number(32), Number(32), Number(32), Number(32), Number(110), Number(97), Number(109), Number(101), Number(58), Number(32), Number(111), Number(120), Number(105), Number(100), Number(101)]}) node_id = 1 saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c
41262025-04-30T06:43:54.510ZDEBGtest_utilization (SEC): saga log event new_state = N002 started sec_id = ac55d8d9-a298-4534-898b-beee1c9e4e52
41272025-04-30T06:43:54.510ZDEBGtest_utilization (SecStore): recording saga event event_type = Started node_id = 2 saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c
41282025-04-30T06:43:54.513ZTRACtest_utilization (SagaExecutor): authorize begin action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true resource = Database saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c saga_name = instance-create saga_node = AssociateSshKeys
41292025-04-30T06:43:54.513ZDEBGtest_utilization (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true roles = RoleSet { roles: {} } saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c saga_name = instance-create saga_node = AssociateSshKeys
41302025-04-30T06:43:54.513ZDEBGtest_utilization (SagaExecutor): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true resource = Database result = Ok(()) saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c saga_name = instance-create saga_node = AssociateSshKeys
41312025-04-30T06:43:54.515ZTRACtest_utilization (SagaExecutor): authorize begin action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true resource = Database saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c saga_name = instance-create saga_node = AssociateSshKeys
41322025-04-30T06:43:54.515ZDEBGtest_utilization (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true roles = RoleSet { roles: {} } saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c saga_name = instance-create saga_node = AssociateSshKeys
41332025-04-30T06:43:54.516ZDEBGtest_utilization (SagaExecutor): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true resource = Database result = Ok(()) saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c saga_name = instance-create saga_node = AssociateSshKeys
41342025-04-30T06:43:54.519ZTRACtest_utilization (SagaExecutor): authorize begin action = ListChildren actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true resource = SiloUser { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000000, lookup_type: ById(001de000-5110-4000-8000-000000000000) }, key: 001de000-05e4-4000-8000-000000004007, lookup_type: ById(001de000-05e4-4000-8000-000000004007) } saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c saga_name = instance-create saga_node = AssociateSshKeys
41352025-04-30T06:43:54.519ZTRACtest_utilization (SagaExecutor): loading roles actor = Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. } actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true resource_id = 001de000-5110-4000-8000-000000000000 resource_type = Silo saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c saga_name = instance-create saga_node = AssociateSshKeys
41362025-04-30T06:43:54.519ZTRACtest_utilization (SagaExecutor): authorize begin action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true resource = Database saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c saga_name = instance-create saga_node = AssociateSshKeys
41372025-04-30T06:43:54.519ZDEBGtest_utilization (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true roles = RoleSet { roles: {} } saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c saga_name = instance-create saga_node = AssociateSshKeys
41382025-04-30T06:43:54.519ZDEBGtest_utilization (SagaExecutor): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true resource = Database result = Ok(()) saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c saga_name = instance-create saga_node = AssociateSshKeys
41392025-04-30T06:43:54.522ZTRACtest_utilization (SagaExecutor): loading roles actor = Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. } actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c saga_name = instance-create saga_node = AssociateSshKeys
41402025-04-30T06:43:54.522ZTRACtest_utilization (SagaExecutor): authorize begin action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true resource = Database saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c saga_name = instance-create saga_node = AssociateSshKeys
41412025-04-30T06:43:54.522ZDEBGtest_utilization (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true roles = RoleSet { roles: {} } saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c saga_name = instance-create saga_node = AssociateSshKeys
41422025-04-30T06:43:54.522ZDEBGtest_utilization (SagaExecutor): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true resource = Database result = Ok(()) saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c saga_name = instance-create saga_node = AssociateSshKeys
41432025-04-30T06:43:54.526ZDEBGtest_utilization (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin"), (Silo, 001de000-5110-4000-8000-000000000000, "admin")} } saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c saga_name = instance-create saga_node = AssociateSshKeys
41442025-04-30T06:43:54.528ZDEBGtest_utilization (SagaExecutor): authorize result action = ListChildren actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true resource = SiloUser { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000000, lookup_type: ById(001de000-5110-4000-8000-000000000000) }, key: 001de000-05e4-4000-8000-000000004007, lookup_type: ById(001de000-05e4-4000-8000-000000004007) } result = Ok(()) saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c saga_name = instance-create saga_node = AssociateSshKeys
41452025-04-30T06:43:54.528ZTRACtest_utilization (SagaExecutor): authorize begin action = ListChildren actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true resource = SiloUser { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000000, lookup_type: ById(001de000-5110-4000-8000-000000000000) }, key: 001de000-05e4-4000-8000-000000004007, lookup_type: ById(001de000-05e4-4000-8000-000000004007) } saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c saga_name = instance-create saga_node = AssociateSshKeys
41462025-04-30T06:43:54.528ZTRACtest_utilization (SagaExecutor): loading roles actor = Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. } actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true resource_id = 001de000-5110-4000-8000-000000000000 resource_type = Silo saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c saga_name = instance-create saga_node = AssociateSshKeys
41472025-04-30T06:43:54.528ZTRACtest_utilization (SagaExecutor): authorize begin action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true resource = Database saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c saga_name = instance-create saga_node = AssociateSshKeys
41482025-04-30T06:43:54.528ZDEBGtest_utilization (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true roles = RoleSet { roles: {} } saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c saga_name = instance-create saga_node = AssociateSshKeys
41492025-04-30T06:43:54.528ZDEBGtest_utilization (SagaExecutor): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true resource = Database result = Ok(()) saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c saga_name = instance-create saga_node = AssociateSshKeys
41502025-04-30T06:43:54.531ZTRACtest_utilization (SagaExecutor): loading roles actor = Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. } actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c saga_name = instance-create saga_node = AssociateSshKeys
41512025-04-30T06:43:54.531ZTRACtest_utilization (SagaExecutor): authorize begin action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true resource = Database saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c saga_name = instance-create saga_node = AssociateSshKeys
41522025-04-30T06:43:54.531ZDEBGtest_utilization (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true roles = RoleSet { roles: {} } saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c saga_name = instance-create saga_node = AssociateSshKeys
41532025-04-30T06:43:54.532ZDEBGtest_utilization (SagaExecutor): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true resource = Database result = Ok(()) saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c saga_name = instance-create saga_node = AssociateSshKeys
41542025-04-30T06:43:54.534ZDEBGtest_utilization (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin"), (Silo, 001de000-5110-4000-8000-000000000000, "admin")} } saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c saga_name = instance-create saga_node = AssociateSshKeys
41552025-04-30T06:43:54.536ZDEBGtest_utilization (SagaExecutor): authorize result action = ListChildren actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true resource = SiloUser { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000000, lookup_type: ById(001de000-5110-4000-8000-000000000000) }, key: 001de000-05e4-4000-8000-000000004007, lookup_type: ById(001de000-05e4-4000-8000-000000004007) } result = Ok(()) saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c saga_name = instance-create saga_node = AssociateSshKeys
41562025-04-30T06:43:54.536ZDEBGtest_utilization (SEC): saga log event new_state = N002 succeeded sec_id = ac55d8d9-a298-4534-898b-beee1c9e4e52
41572025-04-30T06:43:54.536ZDEBGtest_utilization (SecStore): recording saga event event_type = Succeeded(Null) node_id = 2 saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c
41582025-04-30T06:43:54.541ZDEBGtest_utilization (SEC): saga log event new_state = N003 started sec_id = ac55d8d9-a298-4534-898b-beee1c9e4e52
41592025-04-30T06:43:54.541ZDEBGtest_utilization (SecStore): recording saga event event_type = Started node_id = 3 saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c
41602025-04-30T06:43:54.544ZDEBGtest_utilization (SEC): saga log event new_state = N003 succeeded sec_id = ac55d8d9-a298-4534-898b-beee1c9e4e52
41612025-04-30T06:43:54.544ZDEBGtest_utilization (SecStore): recording saga event event_type = Succeeded(Object {"instance_id": String("d7bd2deb-ee27-4661-9e9a-524d734cd6ec"), "new_id": String("64c48731-6cc1-46ff-99f2-86f6c481d0c6"), "saga_params": Object {"boundary_switches": Array [], "create_params": Object {"anti_affinity_groups": Array [], "auto_restart_policy": Null, "boot_disk": Null, "description": String("instance \\"utilization-test-instance\\""), "disks": Array [], "external_ips": Array [], "hostname": String("the-host"), "memory": Number(1073741824), "name": String("utilization-test-instance"), "ncpus": Number(4), "network_interfaces": Object {"type": String("default")}, "ssh_public_keys": Array [], "start": Bool(true), "user_data": String("I2Nsb3VkLWNvbmZpZwpzeXN0ZW1faW5mbzoKICBkZWZhdWx0X3VzZXI6CiAgICBuYW1lOiBveGlkZQ==")}, "project_id": String("473b39f9-8669-45b3-b3d7-793f7e10fc31"), "serialized_authn": Object {"kind": Object {"Authenticated": Array [Object {"actor": Object {"SiloUser": Object {"silo_id": String("001de000-5110-4000-8000-000000000000"), "silo_user_id": String("001de000-05e4-4000-8000-000000004007")}}}, Object {"mapped_fleet_roles": Object {}}]}}}, "which": Number(0)}) node_id = 3 saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c
41622025-04-30T06:43:54.548ZDEBGtest_utilization (SEC): saga log event new_state = N004 started sec_id = ac55d8d9-a298-4534-898b-beee1c9e4e52
41632025-04-30T06:43:54.548ZDEBGtest_utilization (SecStore): recording saga event event_type = Started node_id = 4 saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c
41642025-04-30T06:43:54.551ZDEBGtest_utilization (SEC): saga log event new_state = N004 succeeded sec_id = ac55d8d9-a298-4534-898b-beee1c9e4e52
41652025-04-30T06:43:54.551ZDEBGtest_utilization (SecStore): recording saga event event_type = Succeeded(Null) node_id = 4 saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c
41662025-04-30T06:43:54.554ZDEBGtest_utilization (SEC): saga log event new_state = N005 started sec_id = ac55d8d9-a298-4534-898b-beee1c9e4e52
41672025-04-30T06:43:54.554ZDEBGtest_utilization (SecStore): recording saga event event_type = Started node_id = 5 saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c
41682025-04-30T06:43:54.557ZTRACtest_utilization (SagaExecutor): authorize begin action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true resource = Database saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c saga_name = instance-create saga_node = CreateNetworkInterface0
41692025-04-30T06:43:54.557ZDEBGtest_utilization (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true roles = RoleSet { roles: {} } saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c saga_name = instance-create saga_node = CreateNetworkInterface0
41702025-04-30T06:43:54.557ZDEBGtest_utilization (SagaExecutor): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true resource = Database result = Ok(()) saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c saga_name = instance-create saga_node = CreateNetworkInterface0
41712025-04-30T06:43:54.561ZTRACtest_utilization (SagaExecutor): authorize begin action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true resource = Database saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c saga_name = instance-create saga_node = CreateNetworkInterface0
41722025-04-30T06:43:54.561ZDEBGtest_utilization (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true roles = RoleSet { roles: {} } saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c saga_name = instance-create saga_node = CreateNetworkInterface0
41732025-04-30T06:43:54.561ZDEBGtest_utilization (SagaExecutor): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true resource = Database result = Ok(()) saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c saga_name = instance-create saga_node = CreateNetworkInterface0
41742025-04-30T06:43:54.564ZTRACtest_utilization (SagaExecutor): authorize begin action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true resource = Database saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c saga_name = instance-create saga_node = CreateNetworkInterface0
41752025-04-30T06:43:54.564ZDEBGtest_utilization (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true roles = RoleSet { roles: {} } saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c saga_name = instance-create saga_node = CreateNetworkInterface0
41762025-04-30T06:43:54.564ZDEBGtest_utilization (SagaExecutor): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true resource = Database result = Ok(()) saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c saga_name = instance-create saga_node = CreateNetworkInterface0
41772025-04-30T06:43:54.567ZTRACtest_utilization (SagaExecutor): authorize begin action = CreateChild actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true resource = Instance { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000000, lookup_type: ById(001de000-5110-4000-8000-000000000000) }, key: 473b39f9-8669-45b3-b3d7-793f7e10fc31, lookup_type: ById(473b39f9-8669-45b3-b3d7-793f7e10fc31) }, key: d7bd2deb-ee27-4661-9e9a-524d734cd6ec, lookup_type: ById(d7bd2deb-ee27-4661-9e9a-524d734cd6ec) } saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c saga_name = instance-create saga_node = CreateNetworkInterface0
41782025-04-30T06:43:54.567ZTRACtest_utilization (SagaExecutor): loading roles actor = Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. } actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true resource_id = 473b39f9-8669-45b3-b3d7-793f7e10fc31 resource_type = Project saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c saga_name = instance-create saga_node = CreateNetworkInterface0
41792025-04-30T06:43:54.567ZTRACtest_utilization (SagaExecutor): authorize begin action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true resource = Database saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c saga_name = instance-create saga_node = CreateNetworkInterface0
41802025-04-30T06:43:54.567ZDEBGtest_utilization (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true roles = RoleSet { roles: {} } saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c saga_name = instance-create saga_node = CreateNetworkInterface0
41812025-04-30T06:43:54.568ZDEBGtest_utilization (SagaExecutor): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true resource = Database result = Ok(()) saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c saga_name = instance-create saga_node = CreateNetworkInterface0
41822025-04-30T06:43:54.573ZTRACtest_utilization (SagaExecutor): loading roles actor = Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. } actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true resource_id = 001de000-5110-4000-8000-000000000000 resource_type = Silo saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c saga_name = instance-create saga_node = CreateNetworkInterface0
41832025-04-30T06:43:54.573ZTRACtest_utilization (SagaExecutor): authorize begin action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true resource = Database saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c saga_name = instance-create saga_node = CreateNetworkInterface0
41842025-04-30T06:43:54.573ZDEBGtest_utilization (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true roles = RoleSet { roles: {} } saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c saga_name = instance-create saga_node = CreateNetworkInterface0
41852025-04-30T06:43:54.573ZDEBGtest_utilization (SagaExecutor): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true resource = Database result = Ok(()) saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c saga_name = instance-create saga_node = CreateNetworkInterface0
41862025-04-30T06:43:54.581ZTRACtest_utilization (SagaExecutor): loading roles actor = Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. } actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c saga_name = instance-create saga_node = CreateNetworkInterface0
41872025-04-30T06:43:54.582ZTRACtest_utilization (SagaExecutor): authorize begin action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true resource = Database saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c saga_name = instance-create saga_node = CreateNetworkInterface0
41882025-04-30T06:43:54.582ZDEBGtest_utilization (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true roles = RoleSet { roles: {} } saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c saga_name = instance-create saga_node = CreateNetworkInterface0
41892025-04-30T06:43:54.582ZDEBGtest_utilization (SagaExecutor): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true resource = Database result = Ok(()) saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c saga_name = instance-create saga_node = CreateNetworkInterface0
41902025-04-30T06:43:54.589ZDEBGtest_utilization (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin"), (Silo, 001de000-5110-4000-8000-000000000000, "admin")} } saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c saga_name = instance-create saga_node = CreateNetworkInterface0
41912025-04-30T06:43:54.595ZDEBGtest_utilization (SagaExecutor): authorize result action = CreateChild actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true resource = Instance { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000000, lookup_type: ById(001de000-5110-4000-8000-000000000000) }, key: 473b39f9-8669-45b3-b3d7-793f7e10fc31, lookup_type: ById(473b39f9-8669-45b3-b3d7-793f7e10fc31) }, key: d7bd2deb-ee27-4661-9e9a-524d734cd6ec, lookup_type: ById(d7bd2deb-ee27-4661-9e9a-524d734cd6ec) } result = Ok(()) saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c saga_name = instance-create saga_node = CreateNetworkInterface0
41922025-04-30T06:43:54.596ZTRACtest_utilization (SagaExecutor): authorize begin action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true resource = Database saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c saga_name = instance-create saga_node = CreateNetworkInterface0
41932025-04-30T06:43:54.596ZDEBGtest_utilization (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true roles = RoleSet { roles: {} } saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c saga_name = instance-create saga_node = CreateNetworkInterface0
41942025-04-30T06:43:54.596ZDEBGtest_utilization (SagaExecutor): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true resource = Database result = Ok(()) saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c saga_name = instance-create saga_node = CreateNetworkInterface0
41952025-04-30T06:43:54.607ZTRACtest_utilization (SagaExecutor): authorize begin action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true resource = Database saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c saga_name = instance-create saga_node = CreateNetworkInterface0
41962025-04-30T06:43:54.607ZDEBGtest_utilization (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true roles = RoleSet { roles: {} } saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c saga_name = instance-create saga_node = CreateNetworkInterface0
41972025-04-30T06:43:54.608ZDEBGtest_utilization (SagaExecutor): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true resource = Database result = Ok(()) saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c saga_name = instance-create saga_node = CreateNetworkInterface0
41982025-04-30T06:43:54.615ZTRACtest_utilization (SagaExecutor): authorize begin action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true resource = Database saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c saga_name = instance-create saga_node = CreateNetworkInterface0
41992025-04-30T06:43:54.615ZDEBGtest_utilization (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true roles = RoleSet { roles: {} } saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c saga_name = instance-create saga_node = CreateNetworkInterface0
42002025-04-30T06:43:54.616ZDEBGtest_utilization (SagaExecutor): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true resource = Database result = Ok(()) saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c saga_name = instance-create saga_node = CreateNetworkInterface0
42012025-04-30T06:43:54.632ZTRACtest_utilization (SagaExecutor): authorize begin action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true resource = Database saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c saga_name = instance-create saga_node = CreateNetworkInterface0
42022025-04-30T06:43:54.632ZDEBGtest_utilization (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true roles = RoleSet { roles: {} } saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c saga_name = instance-create saga_node = CreateNetworkInterface0
42032025-04-30T06:43:54.638ZDEBGtest_utilization (SagaExecutor): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true resource = Database result = Ok(()) saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c saga_name = instance-create saga_node = CreateNetworkInterface0
42042025-04-30T06:43:54.641ZTRACtest_utilization (SagaExecutor): authorize begin action = Read actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000000, lookup_type: ById(001de000-5110-4000-8000-000000000000) }, key: 473b39f9-8669-45b3-b3d7-793f7e10fc31, lookup_type: ById(473b39f9-8669-45b3-b3d7-793f7e10fc31) }, key: cda41110-774f-439b-b99b-f715ebb74e33, lookup_type: ByName("default") }, key: be0543da-af63-4252-b6b2-ce2bffd4dd49, lookup_type: ByName("default") } saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c saga_name = instance-create saga_node = CreateNetworkInterface0
42052025-04-30T06:43:54.642ZTRACtest_utilization (SagaExecutor): loading roles actor = Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. } actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true resource_id = 473b39f9-8669-45b3-b3d7-793f7e10fc31 resource_type = Project saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c saga_name = instance-create saga_node = CreateNetworkInterface0
42062025-04-30T06:43:54.642ZTRACtest_utilization (SagaExecutor): authorize begin action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true resource = Database saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c saga_name = instance-create saga_node = CreateNetworkInterface0
42072025-04-30T06:43:54.642ZDEBGtest_utilization (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true roles = RoleSet { roles: {} } saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c saga_name = instance-create saga_node = CreateNetworkInterface0
42082025-04-30T06:43:54.657ZDEBGtest_utilization (SagaExecutor): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true resource = Database result = Ok(()) saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c saga_name = instance-create saga_node = CreateNetworkInterface0
42092025-04-30T06:43:54.694ZTRACtest_utilization (SagaExecutor): loading roles actor = Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. } actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true resource_id = 001de000-5110-4000-8000-000000000000 resource_type = Silo saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c saga_name = instance-create saga_node = CreateNetworkInterface0
42102025-04-30T06:43:54.694ZTRACtest_utilization (SagaExecutor): authorize begin action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true resource = Database saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c saga_name = instance-create saga_node = CreateNetworkInterface0
42112025-04-30T06:43:54.694ZDEBGtest_utilization (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true roles = RoleSet { roles: {} } saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c saga_name = instance-create saga_node = CreateNetworkInterface0
42122025-04-30T06:43:54.694ZDEBGtest_utilization (SagaExecutor): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true resource = Database result = Ok(()) saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c saga_name = instance-create saga_node = CreateNetworkInterface0
42132025-04-30T06:43:54.713ZTRACtest_utilization (SagaExecutor): loading roles actor = Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. } actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c saga_name = instance-create saga_node = CreateNetworkInterface0
42142025-04-30T06:43:54.713ZTRACtest_utilization (SagaExecutor): authorize begin action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true resource = Database saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c saga_name = instance-create saga_node = CreateNetworkInterface0
42152025-04-30T06:43:54.713ZDEBGtest_utilization (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true roles = RoleSet { roles: {} } saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c saga_name = instance-create saga_node = CreateNetworkInterface0
42162025-04-30T06:43:54.714ZDEBGtest_utilization (SagaExecutor): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true resource = Database result = Ok(()) saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c saga_name = instance-create saga_node = CreateNetworkInterface0
42172025-04-30T06:43:54.718ZDEBGtest_utilization (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin"), (Silo, 001de000-5110-4000-8000-000000000000, "admin")} } saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c saga_name = instance-create saga_node = CreateNetworkInterface0
42182025-04-30T06:43:54.727ZDEBGtest_utilization (SagaExecutor): authorize result action = Read actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000000, lookup_type: ById(001de000-5110-4000-8000-000000000000) }, key: 473b39f9-8669-45b3-b3d7-793f7e10fc31, lookup_type: ById(473b39f9-8669-45b3-b3d7-793f7e10fc31) }, key: cda41110-774f-439b-b99b-f715ebb74e33, lookup_type: ByName("default") }, key: be0543da-af63-4252-b6b2-ce2bffd4dd49, lookup_type: ByName("default") } result = Ok(()) saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c saga_name = instance-create saga_node = CreateNetworkInterface0
42192025-04-30T06:43:54.727ZTRACtest_utilization (SagaExecutor): authorize begin action = CreateChild actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true resource = Instance { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000000, lookup_type: ById(001de000-5110-4000-8000-000000000000) }, key: 473b39f9-8669-45b3-b3d7-793f7e10fc31, lookup_type: ById(473b39f9-8669-45b3-b3d7-793f7e10fc31) }, key: d7bd2deb-ee27-4661-9e9a-524d734cd6ec, lookup_type: ById(d7bd2deb-ee27-4661-9e9a-524d734cd6ec) } saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c saga_name = instance-create saga_node = CreateNetworkInterface0
42202025-04-30T06:43:54.728ZTRACtest_utilization (SagaExecutor): loading roles actor = Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. } actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true resource_id = 473b39f9-8669-45b3-b3d7-793f7e10fc31 resource_type = Project saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c saga_name = instance-create saga_node = CreateNetworkInterface0
42212025-04-30T06:43:54.728ZTRACtest_utilization (SagaExecutor): authorize begin action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true resource = Database saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c saga_name = instance-create saga_node = CreateNetworkInterface0
42222025-04-30T06:43:54.728ZDEBGtest_utilization (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true roles = RoleSet { roles: {} } saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c saga_name = instance-create saga_node = CreateNetworkInterface0
42232025-04-30T06:43:54.728ZDEBGtest_utilization (SagaExecutor): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true resource = Database result = Ok(()) saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c saga_name = instance-create saga_node = CreateNetworkInterface0
42242025-04-30T06:43:54.733ZTRACtest_utilization (SagaExecutor): loading roles actor = Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. } actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true resource_id = 001de000-5110-4000-8000-000000000000 resource_type = Silo saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c saga_name = instance-create saga_node = CreateNetworkInterface0
42252025-04-30T06:43:54.733ZTRACtest_utilization (SagaExecutor): authorize begin action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true resource = Database saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c saga_name = instance-create saga_node = CreateNetworkInterface0
42262025-04-30T06:43:54.733ZDEBGtest_utilization (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true roles = RoleSet { roles: {} } saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c saga_name = instance-create saga_node = CreateNetworkInterface0
42272025-04-30T06:43:54.734ZDEBGtest_utilization (SagaExecutor): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true resource = Database result = Ok(()) saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c saga_name = instance-create saga_node = CreateNetworkInterface0
42282025-04-30T06:43:54.743ZTRACtest_utilization (SagaExecutor): loading roles actor = Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. } actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c saga_name = instance-create saga_node = CreateNetworkInterface0
42292025-04-30T06:43:54.743ZTRACtest_utilization (SagaExecutor): authorize begin action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true resource = Database saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c saga_name = instance-create saga_node = CreateNetworkInterface0
42302025-04-30T06:43:54.743ZDEBGtest_utilization (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true roles = RoleSet { roles: {} } saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c saga_name = instance-create saga_node = CreateNetworkInterface0
42312025-04-30T06:43:54.743ZDEBGtest_utilization (SagaExecutor): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true resource = Database result = Ok(()) saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c saga_name = instance-create saga_node = CreateNetworkInterface0
42322025-04-30T06:43:54.746ZDEBGtest_utilization (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin"), (Silo, 001de000-5110-4000-8000-000000000000, "admin")} } saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c saga_name = instance-create saga_node = CreateNetworkInterface0
42332025-04-30T06:43:54.752ZDEBGtest_utilization (SagaExecutor): authorize result action = CreateChild actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true resource = Instance { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000000, lookup_type: ById(001de000-5110-4000-8000-000000000000) }, key: 473b39f9-8669-45b3-b3d7-793f7e10fc31, lookup_type: ById(473b39f9-8669-45b3-b3d7-793f7e10fc31) }, key: d7bd2deb-ee27-4661-9e9a-524d734cd6ec, lookup_type: ById(d7bd2deb-ee27-4661-9e9a-524d734cd6ec) } result = Ok(()) saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c saga_name = instance-create saga_node = CreateNetworkInterface0
42342025-04-30T06:43:54.752ZTRACtest_utilization (SagaExecutor): authorize begin action = CreateChild actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000000, lookup_type: ById(001de000-5110-4000-8000-000000000000) }, key: 473b39f9-8669-45b3-b3d7-793f7e10fc31, lookup_type: ById(473b39f9-8669-45b3-b3d7-793f7e10fc31) }, key: cda41110-774f-439b-b99b-f715ebb74e33, lookup_type: ByName("default") }, key: be0543da-af63-4252-b6b2-ce2bffd4dd49, lookup_type: ByName("default") } saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c saga_name = instance-create saga_node = CreateNetworkInterface0
42352025-04-30T06:43:54.752ZTRACtest_utilization (SagaExecutor): loading roles actor = Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. } actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true resource_id = 473b39f9-8669-45b3-b3d7-793f7e10fc31 resource_type = Project saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c saga_name = instance-create saga_node = CreateNetworkInterface0
42362025-04-30T06:43:54.752ZTRACtest_utilization (SagaExecutor): authorize begin action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true resource = Database saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c saga_name = instance-create saga_node = CreateNetworkInterface0
42372025-04-30T06:43:54.752ZDEBGtest_utilization (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true roles = RoleSet { roles: {} } saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c saga_name = instance-create saga_node = CreateNetworkInterface0
42382025-04-30T06:43:54.752ZDEBGtest_utilization (SagaExecutor): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true resource = Database result = Ok(()) saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c saga_name = instance-create saga_node = CreateNetworkInterface0
42392025-04-30T06:43:54.757ZTRACtest_utilization (SagaExecutor): loading roles actor = Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. } actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true resource_id = 001de000-5110-4000-8000-000000000000 resource_type = Silo saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c saga_name = instance-create saga_node = CreateNetworkInterface0
42402025-04-30T06:43:54.757ZTRACtest_utilization (SagaExecutor): authorize begin action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true resource = Database saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c saga_name = instance-create saga_node = CreateNetworkInterface0
42412025-04-30T06:43:54.757ZDEBGtest_utilization (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true roles = RoleSet { roles: {} } saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c saga_name = instance-create saga_node = CreateNetworkInterface0
42422025-04-30T06:43:54.757ZDEBGtest_utilization (SagaExecutor): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true resource = Database result = Ok(()) saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c saga_name = instance-create saga_node = CreateNetworkInterface0
42432025-04-30T06:43:54.764ZTRACtest_utilization (SagaExecutor): loading roles actor = Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. } actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c saga_name = instance-create saga_node = CreateNetworkInterface0
42442025-04-30T06:43:54.766ZTRACtest_utilization (SagaExecutor): authorize begin action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true resource = Database saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c saga_name = instance-create saga_node = CreateNetworkInterface0
42452025-04-30T06:43:54.766ZDEBGtest_utilization (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true roles = RoleSet { roles: {} } saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c saga_name = instance-create saga_node = CreateNetworkInterface0
42462025-04-30T06:43:54.768ZDEBGtest_utilization (SagaExecutor): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true resource = Database result = Ok(()) saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c saga_name = instance-create saga_node = CreateNetworkInterface0
42472025-04-30T06:43:54.788ZDEBGtest_utilization (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin"), (Silo, 001de000-5110-4000-8000-000000000000, "admin")} } saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c saga_name = instance-create saga_node = CreateNetworkInterface0
42482025-04-30T06:43:54.800ZDEBGtest_utilization (SagaExecutor): authorize result action = CreateChild actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000000, lookup_type: ById(001de000-5110-4000-8000-000000000000) }, key: 473b39f9-8669-45b3-b3d7-793f7e10fc31, lookup_type: ById(473b39f9-8669-45b3-b3d7-793f7e10fc31) }, key: cda41110-774f-439b-b99b-f715ebb74e33, lookup_type: ByName("default") }, key: be0543da-af63-4252-b6b2-ce2bffd4dd49, lookup_type: ByName("default") } result = Ok(()) saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c saga_name = instance-create saga_node = CreateNetworkInterface0
42492025-04-30T06:43:54.800ZTRACtest_utilization (SagaExecutor): authorize begin action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true resource = Database saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c saga_name = instance-create saga_node = CreateNetworkInterface0
42502025-04-30T06:43:54.800ZDEBGtest_utilization (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true roles = RoleSet { roles: {} } saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c saga_name = instance-create saga_node = CreateNetworkInterface0
42512025-04-30T06:43:54.801ZDEBGtest_utilization (SagaExecutor): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true resource = Database result = Ok(()) saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c saga_name = instance-create saga_node = CreateNetworkInterface0
42522025-04-30T06:43:54.916ZTRACtest_utilization (SagaExecutor): authorize begin action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true resource = Database saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c saga_name = instance-create saga_node = CreateNetworkInterface0
42532025-04-30T06:43:54.916ZDEBGtest_utilization (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true roles = RoleSet { roles: {} } saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c saga_name = instance-create saga_node = CreateNetworkInterface0
42542025-04-30T06:43:54.917ZDEBGtest_utilization (SagaExecutor): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true resource = Database result = Ok(()) saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c saga_name = instance-create saga_node = CreateNetworkInterface0
42552025-04-30T06:43:54.922ZDEBGtest_utilization (SEC): saga log event new_state = N005 succeeded sec_id = ac55d8d9-a298-4534-898b-beee1c9e4e52
42562025-04-30T06:43:54.922ZDEBGtest_utilization (SecStore): recording saga event event_type = Succeeded(Null) node_id = 5 saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c
42572025-04-30T06:43:54.927ZDEBGtest_utilization (SEC): saga log event new_state = N006 started sec_id = ac55d8d9-a298-4534-898b-beee1c9e4e52
42582025-04-30T06:43:54.927ZDEBGtest_utilization (SecStore): recording saga event event_type = Started node_id = 6 saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c
42592025-04-30T06:43:54.932ZDEBGtest_utilization (SEC): saga log event new_state = N006 succeeded sec_id = ac55d8d9-a298-4534-898b-beee1c9e4e52
42602025-04-30T06:43:54.932ZDEBGtest_utilization (SecStore): recording saga event event_type = Succeeded(Null) node_id = 6 saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c
42612025-04-30T06:43:54.948ZDEBGtest_utilization (SEC): saga log event new_state = N007 started sec_id = ac55d8d9-a298-4534-898b-beee1c9e4e52
42622025-04-30T06:43:54.948ZDEBGtest_utilization (SecStore): recording saga event event_type = Started node_id = 7 saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c
42632025-04-30T06:43:54.960ZDEBGtest_utilization (SEC): saga log event new_state = N007 succeeded sec_id = ac55d8d9-a298-4534-898b-beee1c9e4e52
42642025-04-30T06:43:54.962ZDEBGtest_utilization (SecStore): recording saga event event_type = Succeeded(Object {"instance_id": String("d7bd2deb-ee27-4661-9e9a-524d734cd6ec"), "new_id": String("79be82ce-957c-439a-b67e-f1dacdb57f1e"), "saga_params": Object {"boundary_switches": Array [], "create_params": Object {"anti_affinity_groups": Array [], "auto_restart_policy": Null, "boot_disk": Null, "description": String("instance \\"utilization-test-instance\\""), "disks": Array [], "external_ips": Array [], "hostname": String("the-host"), "memory": Number(1073741824), "name": String("utilization-test-instance"), "ncpus": Number(4), "network_interfaces": Object {"type": String("default")}, "ssh_public_keys": Array [], "start": Bool(true), "user_data": String("I2Nsb3VkLWNvbmZpZwpzeXN0ZW1faW5mbzoKICBkZWZhdWx0X3VzZXI6CiAgICBuYW1lOiBveGlkZQ==")}, "project_id": String("473b39f9-8669-45b3-b3d7-793f7e10fc31"), "serialized_authn": Object {"kind": Object {"Authenticated": Array [Object {"actor": Object {"SiloUser": Object {"silo_id": String("001de000-5110-4000-8000-000000000000"), "silo_user_id": String("001de000-05e4-4000-8000-000000004007")}}}, Object {"mapped_fleet_roles": Object {}}]}}}, "which": Number(1)}) node_id = 7 saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c
42652025-04-30T06:43:54.974ZDEBGtest_utilization (SEC): saga log event new_state = N008 started sec_id = ac55d8d9-a298-4534-898b-beee1c9e4e52
42662025-04-30T06:43:54.975ZDEBGtest_utilization (SecStore): recording saga event event_type = Started node_id = 8 saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c
42672025-04-30T06:43:54.990ZDEBGtest_utilization (SEC): saga log event new_state = N008 succeeded sec_id = ac55d8d9-a298-4534-898b-beee1c9e4e52
42682025-04-30T06:43:54.990ZDEBGtest_utilization (SecStore): recording saga event event_type = Succeeded(Null) node_id = 8 saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c
42692025-04-30T06:43:55.023ZDEBGtest_utilization (collection-task): sent timer-based collection request to the collection task collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
42702025-04-30T06:43:55.023ZDEBGtest_utilization (collection-task): collecting from producer collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
42712025-04-30T06:43:55.024ZDEBGtest_utilization (SEC): saga log event new_state = N009 started sec_id = ac55d8d9-a298-4534-898b-beee1c9e4e52
42722025-04-30T06:43:55.024ZDEBGtest_utilization (SecStore): recording saga event event_type = Started node_id = 9 saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c
42732025-04-30T06:43:55.026ZTRACtest_utilization (collection-task): sent collection request to producer collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
42742025-04-30T06:43:55.027ZDEBGtest_utilization (collection-task): collected results from producer collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 n_results = 1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
42752025-04-30T06:43:55.028ZTRACtest_utilization (collection-task): forwarded results to main collection loop collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
42762025-04-30T06:43:55.028ZTRACtest_utilization (collection-task): top of inner collection loop, waiting for next request collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
42772025-04-30T06:43:55.029ZDEBGtest_utilization (SEC): saga log event new_state = N009 succeeded sec_id = ac55d8d9-a298-4534-898b-beee1c9e4e52
42782025-04-30T06:43:55.029ZDEBGtest_utilization (SecStore): recording saga event event_type = Succeeded(Null) node_id = 9 saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c
42792025-04-30T06:43:55.053ZDEBGtest_utilization (SEC): saga log event new_state = N010 started sec_id = ac55d8d9-a298-4534-898b-beee1c9e4e52
42802025-04-30T06:43:55.054ZDEBGtest_utilization (SecStore): recording saga event event_type = Started node_id = 10 saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c
42812025-04-30T06:43:55.057ZDEBGtest_utilization (SEC): saga log event new_state = N010 succeeded sec_id = ac55d8d9-a298-4534-898b-beee1c9e4e52
42822025-04-30T06:43:55.057ZDEBGtest_utilization (SecStore): recording saga event event_type = Succeeded(Null) node_id = 10 saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c
42832025-04-30T06:43:55.066ZDEBGtest_utilization (SEC): saga log event new_state = N011 started sec_id = ac55d8d9-a298-4534-898b-beee1c9e4e52
42842025-04-30T06:43:55.066ZDEBGtest_utilization (SecStore): recording saga event event_type = Started node_id = 11 saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c
42852025-04-30T06:43:55.071ZDEBGtest_utilization (SEC): saga log event new_state = N011 succeeded sec_id = ac55d8d9-a298-4534-898b-beee1c9e4e52
42862025-04-30T06:43:55.071ZDEBGtest_utilization (SecStore): recording saga event event_type = Succeeded(Object {"instance_id": String("d7bd2deb-ee27-4661-9e9a-524d734cd6ec"), "new_id": String("628c4c7e-b430-4a2a-bfd4-c1a8a5a74fa6"), "saga_params": Object {"boundary_switches": Array [], "create_params": Object {"anti_affinity_groups": Array [], "auto_restart_policy": Null, "boot_disk": Null, "description": String("instance \\"utilization-test-instance\\""), "disks": Array [], "external_ips": Array [], "hostname": String("the-host"), "memory": Number(1073741824), "name": String("utilization-test-instance"), "ncpus": Number(4), "network_interfaces": Object {"type": String("default")}, "ssh_public_keys": Array [], "start": Bool(true), "user_data": String("I2Nsb3VkLWNvbmZpZwpzeXN0ZW1faW5mbzoKICBkZWZhdWx0X3VzZXI6CiAgICBuYW1lOiBveGlkZQ==")}, "project_id": String("473b39f9-8669-45b3-b3d7-793f7e10fc31"), "serialized_authn": Object {"kind": Object {"Authenticated": Array [Object {"actor": Object {"SiloUser": Object {"silo_id": String("001de000-5110-4000-8000-000000000000"), "silo_user_id": String("001de000-05e4-4000-8000-000000004007")}}}, Object {"mapped_fleet_roles": Object {}}]}}}, "which": Number(2)}) node_id = 11 saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c
42872025-04-30T06:43:55.092ZDEBGtest_utilization (SEC): saga log event new_state = N012 started sec_id = ac55d8d9-a298-4534-898b-beee1c9e4e52
42882025-04-30T06:43:55.092ZDEBGtest_utilization (SecStore): recording saga event event_type = Started node_id = 12 saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c
42892025-04-30T06:43:55.097ZDEBGtest_utilization (SEC): saga log event new_state = N012 succeeded sec_id = ac55d8d9-a298-4534-898b-beee1c9e4e52
42902025-04-30T06:43:55.097ZDEBGtest_utilization (SecStore): recording saga event event_type = Succeeded(Null) node_id = 12 saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c
42912025-04-30T06:43:55.107ZDEBGtest_utilization (SEC): saga log event new_state = N013 started sec_id = ac55d8d9-a298-4534-898b-beee1c9e4e52
42922025-04-30T06:43:55.107ZDEBGtest_utilization (SecStore): recording saga event event_type = Started node_id = 13 saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c
42932025-04-30T06:43:55.119ZDEBGtest_utilization (SEC): saga log event new_state = N013 succeeded sec_id = ac55d8d9-a298-4534-898b-beee1c9e4e52
42942025-04-30T06:43:55.119ZDEBGtest_utilization (SecStore): recording saga event event_type = Succeeded(Null) node_id = 13 saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c
42952025-04-30T06:43:55.155ZDEBGtest_utilization (SEC): saga log event new_state = N014 started sec_id = ac55d8d9-a298-4534-898b-beee1c9e4e52
42962025-04-30T06:43:55.155ZDEBGtest_utilization (SecStore): recording saga event event_type = Started node_id = 14 saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c
42972025-04-30T06:43:55.158ZDEBGtest_utilization (SEC): saga log event new_state = N014 succeeded sec_id = ac55d8d9-a298-4534-898b-beee1c9e4e52
42982025-04-30T06:43:55.158ZDEBGtest_utilization (SecStore): recording saga event event_type = Succeeded(Null) node_id = 14 saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c
42992025-04-30T06:43:55.166ZDEBGtest_utilization (SEC): saga log event new_state = N015 started sec_id = ac55d8d9-a298-4534-898b-beee1c9e4e52
43002025-04-30T06:43:55.166ZDEBGtest_utilization (SecStore): recording saga event event_type = Started node_id = 15 saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c
43012025-04-30T06:43:55.170ZDEBGtest_utilization (SEC): saga log event new_state = N015 succeeded sec_id = ac55d8d9-a298-4534-898b-beee1c9e4e52
43022025-04-30T06:43:55.170ZDEBGtest_utilization (SecStore): recording saga event event_type = Succeeded(Object {"instance_id": String("d7bd2deb-ee27-4661-9e9a-524d734cd6ec"), "new_id": String("9b0302b2-2547-4705-b879-b2d896fa654b"), "saga_params": Object {"boundary_switches": Array [], "create_params": Object {"anti_affinity_groups": Array [], "auto_restart_policy": Null, "boot_disk": Null, "description": String("instance \\"utilization-test-instance\\""), "disks": Array [], "external_ips": Array [], "hostname": String("the-host"), "memory": Number(1073741824), "name": String("utilization-test-instance"), "ncpus": Number(4), "network_interfaces": Object {"type": String("default")}, "ssh_public_keys": Array [], "start": Bool(true), "user_data": String("I2Nsb3VkLWNvbmZpZwpzeXN0ZW1faW5mbzoKICBkZWZhdWx0X3VzZXI6CiAgICBuYW1lOiBveGlkZQ==")}, "project_id": String("473b39f9-8669-45b3-b3d7-793f7e10fc31"), "serialized_authn": Object {"kind": Object {"Authenticated": Array [Object {"actor": Object {"SiloUser": Object {"silo_id": String("001de000-5110-4000-8000-000000000000"), "silo_user_id": String("001de000-05e4-4000-8000-000000004007")}}}, Object {"mapped_fleet_roles": Object {}}]}}}, "which": Number(3)}) node_id = 15 saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c
43032025-04-30T06:43:55.176ZDEBGtest_utilization (SEC): saga log event new_state = N016 started sec_id = ac55d8d9-a298-4534-898b-beee1c9e4e52
43042025-04-30T06:43:55.176ZDEBGtest_utilization (SecStore): recording saga event event_type = Started node_id = 16 saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c
43052025-04-30T06:43:55.196ZDEBGtest_utilization (SEC): saga log event new_state = N016 succeeded sec_id = ac55d8d9-a298-4534-898b-beee1c9e4e52
43062025-04-30T06:43:55.196ZDEBGtest_utilization (SecStore): recording saga event event_type = Succeeded(Null) node_id = 16 saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c
43072025-04-30T06:43:55.201ZDEBGtest_utilization (SEC): saga log event new_state = N017 started sec_id = ac55d8d9-a298-4534-898b-beee1c9e4e52
43082025-04-30T06:43:55.201ZDEBGtest_utilization (SecStore): recording saga event event_type = Started node_id = 17 saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c
43092025-04-30T06:43:55.225ZDEBGtest_utilization (SEC): saga log event new_state = N017 succeeded sec_id = ac55d8d9-a298-4534-898b-beee1c9e4e52
43102025-04-30T06:43:55.232ZDEBGtest_utilization (SecStore): recording saga event event_type = Succeeded(Null) node_id = 17 saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c
43112025-04-30T06:43:55.243ZDEBGtest_utilization (SEC): saga log event new_state = N018 started sec_id = ac55d8d9-a298-4534-898b-beee1c9e4e52
43122025-04-30T06:43:55.243ZDEBGtest_utilization (SecStore): recording saga event event_type = Started node_id = 18 saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c
43132025-04-30T06:43:55.247ZDEBGtest_utilization (SEC): saga log event new_state = N018 succeeded sec_id = ac55d8d9-a298-4534-898b-beee1c9e4e52
43142025-04-30T06:43:55.247ZDEBGtest_utilization (SecStore): recording saga event event_type = Succeeded(Null) node_id = 18 saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c
43152025-04-30T06:43:55.251ZDEBGtest_utilization (SEC): saga log event new_state = N019 started sec_id = ac55d8d9-a298-4534-898b-beee1c9e4e52
43162025-04-30T06:43:55.251ZDEBGtest_utilization (SecStore): recording saga event event_type = Started node_id = 19 saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c
43172025-04-30T06:43:55.259ZDEBGtest_utilization (SEC): saga log event new_state = N019 succeeded sec_id = ac55d8d9-a298-4534-898b-beee1c9e4e52
43182025-04-30T06:43:55.259ZDEBGtest_utilization (SecStore): recording saga event event_type = Succeeded(Object {"instance_id": String("d7bd2deb-ee27-4661-9e9a-524d734cd6ec"), "new_id": String("7390678b-5d44-43e6-b46d-3c25a5eebd71"), "saga_params": Object {"boundary_switches": Array [], "create_params": Object {"anti_affinity_groups": Array [], "auto_restart_policy": Null, "boot_disk": Null, "description": String("instance \\"utilization-test-instance\\""), "disks": Array [], "external_ips": Array [], "hostname": String("the-host"), "memory": Number(1073741824), "name": String("utilization-test-instance"), "ncpus": Number(4), "network_interfaces": Object {"type": String("default")}, "ssh_public_keys": Array [], "start": Bool(true), "user_data": String("I2Nsb3VkLWNvbmZpZwpzeXN0ZW1faW5mbzoKICBkZWZhdWx0X3VzZXI6CiAgICBuYW1lOiBveGlkZQ==")}, "project_id": String("473b39f9-8669-45b3-b3d7-793f7e10fc31"), "serialized_authn": Object {"kind": Object {"Authenticated": Array [Object {"actor": Object {"SiloUser": Object {"silo_id": String("001de000-5110-4000-8000-000000000000"), "silo_user_id": String("001de000-05e4-4000-8000-000000004007")}}}, Object {"mapped_fleet_roles": Object {}}]}}}, "which": Number(4)}) node_id = 19 saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c
43192025-04-30T06:43:55.272ZDEBGtest_utilization (SEC): saga log event new_state = N020 started sec_id = ac55d8d9-a298-4534-898b-beee1c9e4e52
43202025-04-30T06:43:55.272ZDEBGtest_utilization (SecStore): recording saga event event_type = Started node_id = 20 saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c
43212025-04-30T06:43:55.310ZDEBGtest_utilization (SEC): saga log event new_state = N020 succeeded sec_id = ac55d8d9-a298-4534-898b-beee1c9e4e52
43222025-04-30T06:43:55.310ZDEBGtest_utilization (SecStore): recording saga event event_type = Succeeded(Null) node_id = 20 saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c
43232025-04-30T06:43:55.314ZDEBGtest_utilization (SEC): saga log event new_state = N021 started sec_id = ac55d8d9-a298-4534-898b-beee1c9e4e52
43242025-04-30T06:43:55.314ZDEBGtest_utilization (SecStore): recording saga event event_type = Started node_id = 21 saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c
43252025-04-30T06:43:55.319ZDEBGtest_utilization (SEC): saga log event new_state = N021 succeeded sec_id = ac55d8d9-a298-4534-898b-beee1c9e4e52
43262025-04-30T06:43:55.320ZDEBGtest_utilization (SecStore): recording saga event event_type = Succeeded(Null) node_id = 21 saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c
43272025-04-30T06:43:55.340ZDEBGtest_utilization (SEC): saga log event new_state = N022 started sec_id = ac55d8d9-a298-4534-898b-beee1c9e4e52
43282025-04-30T06:43:55.340ZDEBGtest_utilization (SecStore): recording saga event event_type = Started node_id = 22 saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c
43292025-04-30T06:43:55.344ZDEBGtest_utilization (SEC): saga log event new_state = N022 succeeded sec_id = ac55d8d9-a298-4534-898b-beee1c9e4e52
43302025-04-30T06:43:55.346ZDEBGtest_utilization (SecStore): recording saga event event_type = Succeeded(Null) node_id = 22 saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c
43312025-04-30T06:43:55.351ZDEBGtest_utilization (SEC): saga log event new_state = N023 started sec_id = ac55d8d9-a298-4534-898b-beee1c9e4e52
43322025-04-30T06:43:55.351ZDEBGtest_utilization (SecStore): recording saga event event_type = Started node_id = 23 saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c
43332025-04-30T06:43:55.359ZDEBGtest_utilization (results-sink): inserting 1 samples into database collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
43342025-04-30T06:43:55.359ZDEBGtest_utilization (results-sink-cluster): inserting 1 samples into database collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
43352025-04-30T06:43:55.359ZDEBGtest_utilization (clickhouse-client): unrolling 1 total samples collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 4d1485f0-110e-4161-9a33-c4610d68be5d
43362025-04-30T06:43:55.360ZDEBGtest_utilization (clickhouse-client): retrieving timeseries schema from database collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 4d1485f0-110e-4161-9a33-c4610d68be5d
43372025-04-30T06:43:55.360ZTRACtest_utilization (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 4d1485f0-110e-4161-9a33-c4610d68be5d sql = SELECT * FROM oximeter.timeseries_schema FORMAT Native;
43382025-04-30T06:43:55.360ZDEBGtest_utilization (SEC): saga log event new_state = N023 succeeded sec_id = ac55d8d9-a298-4534-898b-beee1c9e4e52
43392025-04-30T06:43:55.360ZDEBGtest_utilization (SecStore): recording saga event event_type = Succeeded(Object {"instance_id": String("d7bd2deb-ee27-4661-9e9a-524d734cd6ec"), "new_id": String("2a09f6d6-3de1-4bea-b89c-6fc00eaf504e"), "saga_params": Object {"boundary_switches": Array [], "create_params": Object {"anti_affinity_groups": Array [], "auto_restart_policy": Null, "boot_disk": Null, "description": String("instance \\"utilization-test-instance\\""), "disks": Array [], "external_ips": Array [], "hostname": String("the-host"), "memory": Number(1073741824), "name": String("utilization-test-instance"), "ncpus": Number(4), "network_interfaces": Object {"type": String("default")}, "ssh_public_keys": Array [], "start": Bool(true), "user_data": String("I2Nsb3VkLWNvbmZpZwpzeXN0ZW1faW5mbzoKICBkZWZhdWx0X3VzZXI6CiAgICBuYW1lOiBveGlkZQ==")}, "project_id": String("473b39f9-8669-45b3-b3d7-793f7e10fc31"), "serialized_authn": Object {"kind": Object {"Authenticated": Array [Object {"actor": Object {"SiloUser": Object {"silo_id": String("001de000-5110-4000-8000-000000000000"), "silo_user_id": String("001de000-05e4-4000-8000-000000004007")}}}, Object {"mapped_fleet_roles": Object {}}]}}}, "which": Number(5)}) node_id = 23 saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c
43402025-04-30T06:43:55.363ZTRACtest_utilization (clickhouse-client): no new timeseries schema in database collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 4d1485f0-110e-4161-9a33-c4610d68be5d
43412025-04-30T06:43:55.363ZDEBGtest_utilization (clickhouse-client): new timeseries schema collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 4d1485f0-110e-4161-9a33-c4610d68be5d schema = TimeseriesSchema { timeseries_name: TimeseriesName("integration_target:integration_metric"), description: TimeseriesDescription { target: "", metric: "" }, field_schema: {FieldSchema { name: "metric_name", field_type: String, source: Metric, description: "" }, FieldSchema { name: "target_name", field_type: String, source: Target, description: "" }}, datum_type: I64, version: 1, authz_scope: Fleet, units: Count, created: 2025-04-30T06:43:55.360103833Z } timeseries_name = integration_target:integration_metric
43422025-04-30T06:43:55.364ZDEBGtest_utilization (clickhouse-client): inserting 1 new timeseries schema collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 4d1485f0-110e-4161-9a33-c4610d68be5d
43432025-04-30T06:43:55.364ZTRACtest_utilization (clickhouse-client): inserting data collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 4d1485f0-110e-4161-9a33-c4610d68be5d n_columns = 6 n_rows = 1 sql = INSERT INTO oximeter.timeseries_schema FORMAT Native
43442025-04-30T06:43:55.366ZDEBGtest_utilization (SEC): saga log event new_state = N024 started sec_id = ac55d8d9-a298-4534-898b-beee1c9e4e52
43452025-04-30T06:43:55.366ZDEBGtest_utilization (SecStore): recording saga event event_type = Started node_id = 24 saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c
43462025-04-30T06:43:55.375ZTRACtest_utilization (clickhouse-client): inserting data collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 4d1485f0-110e-4161-9a33-c4610d68be5d n_columns = 4 n_rows = 2 sql = INSERT INTO oximeter.fields_string FORMAT Native
43472025-04-30T06:43:55.375ZDEBGtest_utilization (SEC): saga log event new_state = N024 succeeded sec_id = ac55d8d9-a298-4534-898b-beee1c9e4e52
43482025-04-30T06:43:55.375ZDEBGtest_utilization (SecStore): recording saga event event_type = Succeeded(Null) node_id = 24 saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c
43492025-04-30T06:43:55.378ZDEBGtest_utilization (SEC): saga log event new_state = N025 started sec_id = ac55d8d9-a298-4534-898b-beee1c9e4e52
43502025-04-30T06:43:55.378ZDEBGtest_utilization (SecStore): recording saga event event_type = Started node_id = 25 saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c
43512025-04-30T06:43:55.386ZDEBGtest_utilization (clickhouse-client): inserted rows into table collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 4d1485f0-110e-4161-9a33-c4610d68be5d n_rows = 2 table_name = fields_string
43522025-04-30T06:43:55.386ZTRACtest_utilization (clickhouse-client): inserting data collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 4d1485f0-110e-4161-9a33-c4610d68be5d n_columns = 4 n_rows = 1 sql = INSERT INTO oximeter.measurements_i64 FORMAT Native
43532025-04-30T06:43:55.391ZDEBGtest_utilization (clickhouse-client): inserted rows into table collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 4d1485f0-110e-4161-9a33-c4610d68be5d n_rows = 1 table_name = measurements_i64
43542025-04-30T06:43:55.391ZTRACtest_utilization (results-sink): successfully inserted samples collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
43552025-04-30T06:43:55.392ZDEBGtest_utilization (SEC): saga log event new_state = N025 succeeded sec_id = ac55d8d9-a298-4534-898b-beee1c9e4e52
43562025-04-30T06:43:55.392ZDEBGtest_utilization (SecStore): recording saga event event_type = Succeeded(Null) node_id = 25 saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c
43572025-04-30T06:43:55.398ZDEBGtest_utilization (SEC): saga log event new_state = N026 started sec_id = ac55d8d9-a298-4534-898b-beee1c9e4e52
43582025-04-30T06:43:55.398ZDEBGtest_utilization (SecStore): recording saga event event_type = Started node_id = 26 saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c
43592025-04-30T06:43:55.407ZDEBGtest_utilization (SEC): saga log event new_state = N026 succeeded sec_id = ac55d8d9-a298-4534-898b-beee1c9e4e52
43602025-04-30T06:43:55.407ZDEBGtest_utilization (SecStore): recording saga event event_type = Succeeded(Null) node_id = 26 saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c
43612025-04-30T06:43:55.415ZDEBGtest_utilization (SEC): saga log event new_state = N027 started sec_id = ac55d8d9-a298-4534-898b-beee1c9e4e52
43622025-04-30T06:43:55.416ZDEBGtest_utilization (SecStore): recording saga event event_type = Started node_id = 27 saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c
43632025-04-30T06:43:55.432ZDEBGtest_utilization (SEC): saga log event new_state = N027 succeeded sec_id = ac55d8d9-a298-4534-898b-beee1c9e4e52
43642025-04-30T06:43:55.432ZDEBGtest_utilization (SecStore): recording saga event event_type = Succeeded(Object {"instance_id": String("d7bd2deb-ee27-4661-9e9a-524d734cd6ec"), "new_id": String("86338488-d577-4ce7-a835-4c7491bc0d88"), "saga_params": Object {"boundary_switches": Array [], "create_params": Object {"anti_affinity_groups": Array [], "auto_restart_policy": Null, "boot_disk": Null, "description": String("instance \\"utilization-test-instance\\""), "disks": Array [], "external_ips": Array [], "hostname": String("the-host"), "memory": Number(1073741824), "name": String("utilization-test-instance"), "ncpus": Number(4), "network_interfaces": Object {"type": String("default")}, "ssh_public_keys": Array [], "start": Bool(true), "user_data": String("I2Nsb3VkLWNvbmZpZwpzeXN0ZW1faW5mbzoKICBkZWZhdWx0X3VzZXI6CiAgICBuYW1lOiBveGlkZQ==")}, "project_id": String("473b39f9-8669-45b3-b3d7-793f7e10fc31"), "serialized_authn": Object {"kind": Object {"Authenticated": Array [Object {"actor": Object {"SiloUser": Object {"silo_id": String("001de000-5110-4000-8000-000000000000"), "silo_user_id": String("001de000-05e4-4000-8000-000000004007")}}}, Object {"mapped_fleet_roles": Object {}}]}}}, "which": Number(6)}) node_id = 27 saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c
43652025-04-30T06:43:55.437ZDEBGtest_utilization (SEC): saga log event new_state = N028 started sec_id = ac55d8d9-a298-4534-898b-beee1c9e4e52
43662025-04-30T06:43:55.440ZDEBGtest_utilization (SecStore): recording saga event event_type = Started node_id = 28 saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c
43672025-04-30T06:43:55.443ZDEBGtest_utilization (SEC): saga log event new_state = N028 succeeded sec_id = ac55d8d9-a298-4534-898b-beee1c9e4e52
43682025-04-30T06:43:55.443ZDEBGtest_utilization (SecStore): recording saga event event_type = Succeeded(Null) node_id = 28 saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c
43692025-04-30T06:43:55.460ZWARNtest_utilization (results-sink-cluster): failed to insert some results into metric DB collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 error = Failed to check out connection to database: No backends found for this service file = oximeter/collector/src/results_sink.rs:92
43702025-04-30T06:43:55.461ZDEBGtest_utilization (SEC): saga log event new_state = N029 started sec_id = ac55d8d9-a298-4534-898b-beee1c9e4e52
43712025-04-30T06:43:55.461ZDEBGtest_utilization (SecStore): recording saga event event_type = Started node_id = 29 saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c
43722025-04-30T06:43:55.465ZDEBGtest_utilization (SEC): saga log event new_state = N029 succeeded sec_id = ac55d8d9-a298-4534-898b-beee1c9e4e52
43732025-04-30T06:43:55.465ZDEBGtest_utilization (SecStore): recording saga event event_type = Succeeded(Null) node_id = 29 saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c
43742025-04-30T06:43:55.470ZDEBGtest_utilization (SEC): saga log event new_state = N030 started sec_id = ac55d8d9-a298-4534-898b-beee1c9e4e52
43752025-04-30T06:43:55.470ZDEBGtest_utilization (SecStore): recording saga event event_type = Started node_id = 30 saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c
43762025-04-30T06:43:55.473ZDEBGtest_utilization (SEC): saga log event new_state = N030 succeeded sec_id = ac55d8d9-a298-4534-898b-beee1c9e4e52
43772025-04-30T06:43:55.473ZDEBGtest_utilization (SecStore): recording saga event event_type = Succeeded(Null) node_id = 30 saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c
43782025-04-30T06:43:55.477ZDEBGtest_utilization (SEC): saga log event new_state = N031 started sec_id = ac55d8d9-a298-4534-898b-beee1c9e4e52
43792025-04-30T06:43:55.477ZDEBGtest_utilization (SecStore): recording saga event event_type = Started node_id = 31 saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c
43802025-04-30T06:43:55.480ZDEBGtest_utilization (SEC): saga log event new_state = N031 succeeded sec_id = ac55d8d9-a298-4534-898b-beee1c9e4e52
43812025-04-30T06:43:55.480ZDEBGtest_utilization (SecStore): recording saga event event_type = Succeeded(Object {"instance_id": String("d7bd2deb-ee27-4661-9e9a-524d734cd6ec"), "new_id": String("d7156580-5c17-44d4-bbc0-c78d1b1133df"), "saga_params": Object {"boundary_switches": Array [], "create_params": Object {"anti_affinity_groups": Array [], "auto_restart_policy": Null, "boot_disk": Null, "description": String("instance \\"utilization-test-instance\\""), "disks": Array [], "external_ips": Array [], "hostname": String("the-host"), "memory": Number(1073741824), "name": String("utilization-test-instance"), "ncpus": Number(4), "network_interfaces": Object {"type": String("default")}, "ssh_public_keys": Array [], "start": Bool(true), "user_data": String("I2Nsb3VkLWNvbmZpZwpzeXN0ZW1faW5mbzoKICBkZWZhdWx0X3VzZXI6CiAgICBuYW1lOiBveGlkZQ==")}, "project_id": String("473b39f9-8669-45b3-b3d7-793f7e10fc31"), "serialized_authn": Object {"kind": Object {"Authenticated": Array [Object {"actor": Object {"SiloUser": Object {"silo_id": String("001de000-5110-4000-8000-000000000000"), "silo_user_id": String("001de000-05e4-4000-8000-000000004007")}}}, Object {"mapped_fleet_roles": Object {}}]}}}, "which": Number(7)}) node_id = 31 saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c
43822025-04-30T06:43:55.483ZDEBGtest_utilization (SEC): saga log event new_state = N032 started sec_id = ac55d8d9-a298-4534-898b-beee1c9e4e52
43832025-04-30T06:43:55.483ZDEBGtest_utilization (SecStore): recording saga event event_type = Started node_id = 32 saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c
43842025-04-30T06:43:55.486ZDEBGtest_utilization (SEC): saga log event new_state = N032 succeeded sec_id = ac55d8d9-a298-4534-898b-beee1c9e4e52
43852025-04-30T06:43:55.486ZDEBGtest_utilization (SecStore): recording saga event event_type = Succeeded(Null) node_id = 32 saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c
43862025-04-30T06:43:55.490ZDEBGtest_utilization (SEC): saga log event new_state = N033 started sec_id = ac55d8d9-a298-4534-898b-beee1c9e4e52
43872025-04-30T06:43:55.490ZDEBGtest_utilization (SecStore): recording saga event event_type = Started node_id = 33 saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c
43882025-04-30T06:43:55.493ZDEBGtest_utilization (SEC): saga log event new_state = N033 succeeded sec_id = ac55d8d9-a298-4534-898b-beee1c9e4e52
43892025-04-30T06:43:55.493ZDEBGtest_utilization (SecStore): recording saga event event_type = Succeeded(Null) node_id = 33 saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c
43902025-04-30T06:43:55.499ZDEBGtest_utilization (SEC): saga log event new_state = N034 started sec_id = ac55d8d9-a298-4534-898b-beee1c9e4e52
43912025-04-30T06:43:55.499ZDEBGtest_utilization (SecStore): recording saga event event_type = Started node_id = 34 saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c
43922025-04-30T06:43:55.502ZDEBGtest_utilization (SEC): saga log event new_state = N034 succeeded sec_id = ac55d8d9-a298-4534-898b-beee1c9e4e52
43932025-04-30T06:43:55.502ZDEBGtest_utilization (SecStore): recording saga event event_type = Succeeded(Null) node_id = 34 saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c
43942025-04-30T06:43:55.506ZDEBGtest_utilization (SEC): saga log event new_state = N035 started sec_id = ac55d8d9-a298-4534-898b-beee1c9e4e52
43952025-04-30T06:43:55.506ZDEBGtest_utilization (SecStore): recording saga event event_type = Started node_id = 35 saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c
43962025-04-30T06:43:55.511ZDEBGtest_utilization (SEC): saga log event new_state = N035 succeeded sec_id = ac55d8d9-a298-4534-898b-beee1c9e4e52
43972025-04-30T06:43:55.512ZDEBGtest_utilization (SecStore): recording saga event event_type = Succeeded(String("703ac9cc-cf3f-44f5-9e01-0a385ec2b26a")) node_id = 35 saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c
43982025-04-30T06:43:55.515ZDEBGtest_utilization (SEC): saga log event new_state = N036 started sec_id = ac55d8d9-a298-4534-898b-beee1c9e4e52
43992025-04-30T06:43:55.515ZDEBGtest_utilization (SecStore): recording saga event event_type = Started node_id = 36 saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c
44002025-04-30T06:43:55.518ZTRACtest_utilization (SagaExecutor): authorize begin action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true resource = Database saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c saga_name = instance-create saga_node = CreateSnatIp
44012025-04-30T06:43:55.518ZDEBGtest_utilization (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true roles = RoleSet { roles: {} } saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c saga_name = instance-create saga_node = CreateSnatIp
44022025-04-30T06:43:55.519ZDEBGtest_utilization (SagaExecutor): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true resource = Database result = Ok(()) saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c saga_name = instance-create saga_node = CreateSnatIp
44032025-04-30T06:43:55.522ZTRACtest_utilization (SagaExecutor): authorize begin action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true resource = Database saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c saga_name = instance-create saga_node = CreateSnatIp
44042025-04-30T06:43:55.522ZDEBGtest_utilization (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true roles = RoleSet { roles: {} } saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c saga_name = instance-create saga_node = CreateSnatIp
44052025-04-30T06:43:55.522ZDEBGtest_utilization (SagaExecutor): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true resource = Database result = Ok(()) saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c saga_name = instance-create saga_node = CreateSnatIp
44062025-04-30T06:43:55.743ZDEBGtest_utilization (SEC): saga log event new_state = N036 succeeded sec_id = ac55d8d9-a298-4534-898b-beee1c9e4e52
44072025-04-30T06:43:55.743ZDEBGtest_utilization (SecStore): recording saga event event_type = Succeeded(Null) node_id = 36 saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c
44082025-04-30T06:43:55.748ZDEBGtest_utilization (SEC): saga log event new_state = N037 started sec_id = ac55d8d9-a298-4534-898b-beee1c9e4e52
44092025-04-30T06:43:55.748ZDEBGtest_utilization (SecStore): recording saga event event_type = Started node_id = 37 saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c
44102025-04-30T06:43:55.751ZDEBGtest_utilization (SEC): saga log event new_state = N037 succeeded sec_id = ac55d8d9-a298-4534-898b-beee1c9e4e52
44112025-04-30T06:43:55.751ZDEBGtest_utilization (SecStore): recording saga event event_type = Succeeded(Object {"instance_id": String("d7bd2deb-ee27-4661-9e9a-524d734cd6ec"), "new_id": String("d135d564-8cd2-4690-a1c9-1629e2d7c21e"), "saga_params": Object {"boundary_switches": Array [], "create_params": Object {"anti_affinity_groups": Array [], "auto_restart_policy": Null, "boot_disk": Null, "description": String("instance \\"utilization-test-instance\\""), "disks": Array [], "external_ips": Array [], "hostname": String("the-host"), "memory": Number(1073741824), "name": String("utilization-test-instance"), "ncpus": Number(4), "network_interfaces": Object {"type": String("default")}, "ssh_public_keys": Array [], "start": Bool(true), "user_data": String("I2Nsb3VkLWNvbmZpZwpzeXN0ZW1faW5mbzoKICBkZWZhdWx0X3VzZXI6CiAgICBuYW1lOiBveGlkZQ==")}, "project_id": String("473b39f9-8669-45b3-b3d7-793f7e10fc31"), "serialized_authn": Object {"kind": Object {"Authenticated": Array [Object {"actor": Object {"SiloUser": Object {"silo_id": String("001de000-5110-4000-8000-000000000000"), "silo_user_id": String("001de000-05e4-4000-8000-000000004007")}}}, Object {"mapped_fleet_roles": Object {}}]}}}, "which": Number(0)}) node_id = 37 saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c
44122025-04-30T06:43:55.756ZDEBGtest_utilization (SEC): saga log event new_state = N038 started sec_id = ac55d8d9-a298-4534-898b-beee1c9e4e52
44132025-04-30T06:43:55.756ZDEBGtest_utilization (SecStore): recording saga event event_type = Started node_id = 38 saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c
44142025-04-30T06:43:55.772ZDEBGtest_utilization (SEC): saga log event new_state = N038 succeeded sec_id = ac55d8d9-a298-4534-898b-beee1c9e4e52
44152025-04-30T06:43:55.772ZDEBGtest_utilization (SecStore): recording saga event event_type = Succeeded(Null) node_id = 38 saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c
44162025-04-30T06:43:55.776ZDEBGtest_utilization (SEC): saga log event new_state = N039 started sec_id = ac55d8d9-a298-4534-898b-beee1c9e4e52
44172025-04-30T06:43:55.777ZDEBGtest_utilization (SecStore): recording saga event event_type = Started node_id = 39 saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c
44182025-04-30T06:43:55.790ZDEBGtest_utilization (SEC): saga log event new_state = N039 succeeded sec_id = ac55d8d9-a298-4534-898b-beee1c9e4e52
44192025-04-30T06:43:55.790ZDEBGtest_utilization (SecStore): recording saga event event_type = Succeeded(Null) node_id = 39 saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c
44202025-04-30T06:43:55.794ZDEBGtest_utilization (SEC): saga log event new_state = N040 started sec_id = ac55d8d9-a298-4534-898b-beee1c9e4e52
44212025-04-30T06:43:55.794ZDEBGtest_utilization (SecStore): recording saga event event_type = Started node_id = 40 saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c
44222025-04-30T06:43:55.797ZDEBGtest_utilization (SEC): saga log event new_state = N040 succeeded sec_id = ac55d8d9-a298-4534-898b-beee1c9e4e52
44232025-04-30T06:43:55.797ZDEBGtest_utilization (SecStore): recording saga event event_type = Succeeded(Null) node_id = 40 saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c
44242025-04-30T06:43:55.799ZDEBGtest_utilization (SEC): saga log event new_state = N041 started sec_id = ac55d8d9-a298-4534-898b-beee1c9e4e52
44252025-04-30T06:43:55.799ZDEBGtest_utilization (SecStore): recording saga event event_type = Started node_id = 41 saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c
44262025-04-30T06:43:55.802ZDEBGtest_utilization (SEC): saga log event new_state = N041 succeeded sec_id = ac55d8d9-a298-4534-898b-beee1c9e4e52
44272025-04-30T06:43:55.802ZDEBGtest_utilization (SecStore): recording saga event event_type = Succeeded(Object {"instance_id": String("d7bd2deb-ee27-4661-9e9a-524d734cd6ec"), "new_id": String("60556df4-fdfb-48b6-8878-e6da4a1201e5"), "saga_params": Object {"boundary_switches": Array [], "create_params": Object {"anti_affinity_groups": Array [], "auto_restart_policy": Null, "boot_disk": Null, "description": String("instance \\"utilization-test-instance\\""), "disks": Array [], "external_ips": Array [], "hostname": String("the-host"), "memory": Number(1073741824), "name": String("utilization-test-instance"), "ncpus": Number(4), "network_interfaces": Object {"type": String("default")}, "ssh_public_keys": Array [], "start": Bool(true), "user_data": String("I2Nsb3VkLWNvbmZpZwpzeXN0ZW1faW5mbzoKICBkZWZhdWx0X3VzZXI6CiAgICBuYW1lOiBveGlkZQ==")}, "project_id": String("473b39f9-8669-45b3-b3d7-793f7e10fc31"), "serialized_authn": Object {"kind": Object {"Authenticated": Array [Object {"actor": Object {"SiloUser": Object {"silo_id": String("001de000-5110-4000-8000-000000000000"), "silo_user_id": String("001de000-05e4-4000-8000-000000004007")}}}, Object {"mapped_fleet_roles": Object {}}]}}}, "which": Number(1)}) node_id = 41 saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c
44282025-04-30T06:43:55.806ZDEBGtest_utilization (SEC): saga log event new_state = N042 started sec_id = ac55d8d9-a298-4534-898b-beee1c9e4e52
44292025-04-30T06:43:55.806ZDEBGtest_utilization (SecStore): recording saga event event_type = Started node_id = 42 saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c
44302025-04-30T06:43:55.809ZDEBGtest_utilization (SEC): saga log event new_state = N042 succeeded sec_id = ac55d8d9-a298-4534-898b-beee1c9e4e52
44312025-04-30T06:43:55.809ZDEBGtest_utilization (SecStore): recording saga event event_type = Succeeded(Null) node_id = 42 saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c
44322025-04-30T06:43:55.812ZDEBGtest_utilization (SEC): saga log event new_state = N043 started sec_id = ac55d8d9-a298-4534-898b-beee1c9e4e52
44332025-04-30T06:43:55.812ZDEBGtest_utilization (SecStore): recording saga event event_type = Started node_id = 43 saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c
44342025-04-30T06:43:55.816ZDEBGtest_utilization (SEC): saga log event new_state = N043 succeeded sec_id = ac55d8d9-a298-4534-898b-beee1c9e4e52
44352025-04-30T06:43:55.816ZDEBGtest_utilization (SecStore): recording saga event event_type = Succeeded(Null) node_id = 43 saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c
44362025-04-30T06:43:55.820ZDEBGtest_utilization (SEC): saga log event new_state = N044 started sec_id = ac55d8d9-a298-4534-898b-beee1c9e4e52
44372025-04-30T06:43:55.820ZDEBGtest_utilization (SecStore): recording saga event event_type = Started node_id = 44 saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c
44382025-04-30T06:43:55.822ZDEBGtest_utilization (SEC): saga log event new_state = N044 succeeded sec_id = ac55d8d9-a298-4534-898b-beee1c9e4e52
44392025-04-30T06:43:55.822ZDEBGtest_utilization (SecStore): recording saga event event_type = Succeeded(Null) node_id = 44 saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c
44402025-04-30T06:43:55.825ZDEBGtest_utilization (SEC): saga log event new_state = N045 started sec_id = ac55d8d9-a298-4534-898b-beee1c9e4e52
44412025-04-30T06:43:55.825ZDEBGtest_utilization (SecStore): recording saga event event_type = Started node_id = 45 saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c
44422025-04-30T06:43:55.828ZDEBGtest_utilization (SEC): saga log event new_state = N045 succeeded sec_id = ac55d8d9-a298-4534-898b-beee1c9e4e52
44432025-04-30T06:43:55.829ZDEBGtest_utilization (SecStore): recording saga event event_type = Succeeded(Object {"instance_id": String("d7bd2deb-ee27-4661-9e9a-524d734cd6ec"), "new_id": String("95e2067e-7259-4904-bcc6-199c85e45b17"), "saga_params": Object {"boundary_switches": Array [], "create_params": Object {"anti_affinity_groups": Array [], "auto_restart_policy": Null, "boot_disk": Null, "description": String("instance \\"utilization-test-instance\\""), "disks": Array [], "external_ips": Array [], "hostname": String("the-host"), "memory": Number(1073741824), "name": String("utilization-test-instance"), "ncpus": Number(4), "network_interfaces": Object {"type": String("default")}, "ssh_public_keys": Array [], "start": Bool(true), "user_data": String("I2Nsb3VkLWNvbmZpZwpzeXN0ZW1faW5mbzoKICBkZWZhdWx0X3VzZXI6CiAgICBuYW1lOiBveGlkZQ==")}, "project_id": String("473b39f9-8669-45b3-b3d7-793f7e10fc31"), "serialized_authn": Object {"kind": Object {"Authenticated": Array [Object {"actor": Object {"SiloUser": Object {"silo_id": String("001de000-5110-4000-8000-000000000000"), "silo_user_id": String("001de000-05e4-4000-8000-000000004007")}}}, Object {"mapped_fleet_roles": Object {}}]}}}, "which": Number(2)}) node_id = 45 saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c
44442025-04-30T06:43:55.832ZDEBGtest_utilization (SEC): saga log event new_state = N046 started sec_id = ac55d8d9-a298-4534-898b-beee1c9e4e52
44452025-04-30T06:43:55.832ZDEBGtest_utilization (SecStore): recording saga event event_type = Started node_id = 46 saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c
44462025-04-30T06:43:55.835ZDEBGtest_utilization (SEC): saga log event new_state = N046 succeeded sec_id = ac55d8d9-a298-4534-898b-beee1c9e4e52
44472025-04-30T06:43:55.835ZDEBGtest_utilization (SecStore): recording saga event event_type = Succeeded(Null) node_id = 46 saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c
44482025-04-30T06:43:55.847ZDEBGtest_utilization (SEC): saga log event new_state = N047 started sec_id = ac55d8d9-a298-4534-898b-beee1c9e4e52
44492025-04-30T06:43:55.847ZDEBGtest_utilization (SecStore): recording saga event event_type = Started node_id = 47 saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c
44502025-04-30T06:43:55.851ZDEBGtest_utilization (SEC): saga log event new_state = N047 succeeded sec_id = ac55d8d9-a298-4534-898b-beee1c9e4e52
44512025-04-30T06:43:55.851ZDEBGtest_utilization (SecStore): recording saga event event_type = Succeeded(Null) node_id = 47 saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c
44522025-04-30T06:43:55.857ZDEBGtest_utilization (SEC): saga log event new_state = N048 started sec_id = ac55d8d9-a298-4534-898b-beee1c9e4e52
44532025-04-30T06:43:55.857ZDEBGtest_utilization (SecStore): recording saga event event_type = Started node_id = 48 saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c
44542025-04-30T06:43:55.862ZDEBGtest_utilization (SEC): saga log event new_state = N048 succeeded sec_id = ac55d8d9-a298-4534-898b-beee1c9e4e52
44552025-04-30T06:43:55.862ZDEBGtest_utilization (SecStore): recording saga event event_type = Succeeded(Null) node_id = 48 saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c
44562025-04-30T06:43:55.865ZDEBGtest_utilization (SEC): saga log event new_state = N049 started sec_id = ac55d8d9-a298-4534-898b-beee1c9e4e52
44572025-04-30T06:43:55.865ZDEBGtest_utilization (SecStore): recording saga event event_type = Started node_id = 49 saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c
44582025-04-30T06:43:55.868ZDEBGtest_utilization (SEC): saga log event new_state = N049 succeeded sec_id = ac55d8d9-a298-4534-898b-beee1c9e4e52
44592025-04-30T06:43:55.868ZDEBGtest_utilization (SecStore): recording saga event event_type = Succeeded(Object {"instance_id": String("d7bd2deb-ee27-4661-9e9a-524d734cd6ec"), "new_id": String("fdb5b695-fb92-422f-b187-3e8141c4416b"), "saga_params": Object {"boundary_switches": Array [], "create_params": Object {"anti_affinity_groups": Array [], "auto_restart_policy": Null, "boot_disk": Null, "description": String("instance \\"utilization-test-instance\\""), "disks": Array [], "external_ips": Array [], "hostname": String("the-host"), "memory": Number(1073741824), "name": String("utilization-test-instance"), "ncpus": Number(4), "network_interfaces": Object {"type": String("default")}, "ssh_public_keys": Array [], "start": Bool(true), "user_data": String("I2Nsb3VkLWNvbmZpZwpzeXN0ZW1faW5mbzoKICBkZWZhdWx0X3VzZXI6CiAgICBuYW1lOiBveGlkZQ==")}, "project_id": String("473b39f9-8669-45b3-b3d7-793f7e10fc31"), "serialized_authn": Object {"kind": Object {"Authenticated": Array [Object {"actor": Object {"SiloUser": Object {"silo_id": String("001de000-5110-4000-8000-000000000000"), "silo_user_id": String("001de000-05e4-4000-8000-000000004007")}}}, Object {"mapped_fleet_roles": Object {}}]}}}, "which": Number(3)}) node_id = 49 saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c
44602025-04-30T06:43:55.886ZDEBGtest_utilization (SEC): saga log event new_state = N050 started sec_id = ac55d8d9-a298-4534-898b-beee1c9e4e52
44612025-04-30T06:43:55.886ZDEBGtest_utilization (SecStore): recording saga event event_type = Started node_id = 50 saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c
44622025-04-30T06:43:55.907ZDEBGtest_utilization (SEC): saga log event new_state = N050 succeeded sec_id = ac55d8d9-a298-4534-898b-beee1c9e4e52
44632025-04-30T06:43:55.909ZDEBGtest_utilization (SecStore): recording saga event event_type = Succeeded(Null) node_id = 50 saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c
44642025-04-30T06:43:55.956ZDEBGtest_utilization (SEC): saga log event new_state = N051 started sec_id = ac55d8d9-a298-4534-898b-beee1c9e4e52
44652025-04-30T06:43:55.956ZDEBGtest_utilization (SecStore): recording saga event event_type = Started node_id = 51 saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c
44662025-04-30T06:43:55.956ZINFOtest_utilization (oximeter-agent): refreshing list of producers from Nexus collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 file = oximeter/collector/src/agent.rs:562
44672025-04-30T06:43:55.957ZDEBGtest_utilization (oximeter): client request body = None method = GET uri = http://127.0.0.1:32956/metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?sort_by=id_ascending
44682025-04-30T06:43:55.957ZTRACtest_utilization (dropshot_internal): incoming request local_addr = 127.0.0.1:32956 method = GET remote_addr = 127.0.0.1:55797 req_id = f44ea294-b4d3-4217-80aa-7e1f96873be0 uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?sort_by=id_ascending
44692025-04-30T06:43:55.958ZTRACtest_utilization (dropshot_internal): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = 127.0.0.1:32956 method = GET remote_addr = 127.0.0.1:55797 req_id = f44ea294-b4d3-4217-80aa-7e1f96873be0 resource = Database uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?sort_by=id_ascending
44702025-04-30T06:43:55.958ZDEBGtest_utilization (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = 127.0.0.1:32956 method = GET remote_addr = 127.0.0.1:55797 req_id = f44ea294-b4d3-4217-80aa-7e1f96873be0 roles = RoleSet { roles: {} } uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?sort_by=id_ascending
44712025-04-30T06:43:55.959ZDEBGtest_utilization (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = 127.0.0.1:32956 method = GET remote_addr = 127.0.0.1:55797 req_id = f44ea294-b4d3-4217-80aa-7e1f96873be0 resource = Database result = Ok(()) uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?sort_by=id_ascending
44722025-04-30T06:43:55.960ZDEBGtest_utilization (SEC): saga log event new_state = N051 succeeded sec_id = ac55d8d9-a298-4534-898b-beee1c9e4e52
44732025-04-30T06:43:55.960ZDEBGtest_utilization (SecStore): recording saga event event_type = Succeeded(Null) node_id = 51 saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c
44742025-04-30T06:43:55.965ZINFOtest_utilization (dropshot_internal): request completed latency_us = 7895 local_addr = 127.0.0.1:32956 method = GET remote_addr = 127.0.0.1:55797 req_id = f44ea294-b4d3-4217-80aa-7e1f96873be0 response_code = 200 uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?sort_by=id_ascending
44752025-04-30T06:43:55.966ZDEBGtest_utilization (oximeter): client response result = Ok(Response { url: "http://127.0.0.1:32956/metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?sort_by=id_ascending", status: 200, headers: {"content-type": "application/json", "x-request-id": "f44ea294-b4d3-4217-80aa-7e1f96873be0", "content-length": "405", "date": "Wed, 30 Apr 2025 06:43:55 GMT"} })
44762025-04-30T06:43:55.966ZDEBGtest_utilization (oximeter): client request body = None method = GET uri = http://127.0.0.1:32956/metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJhYzU1ZDhkOS1hMjk4LTQ1MzQtODk4Yi1iZWVlMWM5ZTRlNTIifX0%3D
44772025-04-30T06:43:55.972ZDEBGtest_utilization (SEC): saga log event new_state = N052 started sec_id = ac55d8d9-a298-4534-898b-beee1c9e4e52
44782025-04-30T06:43:55.972ZDEBGtest_utilization (SecStore): recording saga event event_type = Started node_id = 52 saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c
44792025-04-30T06:43:55.973ZTRACtest_utilization (dropshot_internal): incoming request local_addr = 127.0.0.1:32956 method = GET remote_addr = 127.0.0.1:55797 req_id = c396fc3f-2bf6-41a4-8cc2-1905f0f079dd uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJhYzU1ZDhkOS1hMjk4LTQ1MzQtODk4Yi1iZWVlMWM5ZTRlNTIifX0%3D
44802025-04-30T06:43:55.973ZTRACtest_utilization (dropshot_internal): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = 127.0.0.1:32956 method = GET remote_addr = 127.0.0.1:55797 req_id = c396fc3f-2bf6-41a4-8cc2-1905f0f079dd resource = Database uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJhYzU1ZDhkOS1hMjk4LTQ1MzQtODk4Yi1iZWVlMWM5ZTRlNTIifX0%3D
44812025-04-30T06:43:55.973ZDEBGtest_utilization (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = 127.0.0.1:32956 method = GET remote_addr = 127.0.0.1:55797 req_id = c396fc3f-2bf6-41a4-8cc2-1905f0f079dd roles = RoleSet { roles: {} } uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJhYzU1ZDhkOS1hMjk4LTQ1MzQtODk4Yi1iZWVlMWM5ZTRlNTIifX0%3D
44822025-04-30T06:43:55.974ZDEBGtest_utilization (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = 127.0.0.1:32956 method = GET remote_addr = 127.0.0.1:55797 req_id = c396fc3f-2bf6-41a4-8cc2-1905f0f079dd resource = Database result = Ok(()) uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJhYzU1ZDhkOS1hMjk4LTQ1MzQtODk4Yi1iZWVlMWM5ZTRlNTIifX0%3D
44832025-04-30T06:43:55.980ZDEBGtest_utilization (SEC): saga log event new_state = N052 succeeded sec_id = ac55d8d9-a298-4534-898b-beee1c9e4e52
44842025-04-30T06:43:55.980ZDEBGtest_utilization (SecStore): recording saga event event_type = Succeeded(Null) node_id = 52 saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c
44852025-04-30T06:43:55.983ZINFOtest_utilization (dropshot_internal): request completed latency_us = 10027 local_addr = 127.0.0.1:32956 method = GET remote_addr = 127.0.0.1:55797 req_id = c396fc3f-2bf6-41a4-8cc2-1905f0f079dd response_code = 200 uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJhYzU1ZDhkOS1hMjk4LTQ1MzQtODk4Yi1iZWVlMWM5ZTRlNTIifX0%3D
44862025-04-30T06:43:55.984ZDEBGtest_utilization (oximeter): client response result = Ok(Response { url: "http://127.0.0.1:32956/metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJhYzU1ZDhkOS1hMjk4LTQ1MzQtODk4Yi1iZWVlMWM5ZTRlNTIifX0%3D", status: 200, headers: {"content-type": "application/json", "x-request-id": "c396fc3f-2bf6-41a4-8cc2-1905f0f079dd", "content-length": "29", "date": "Wed, 30 Apr 2025 06:43:55 GMT"} })
44872025-04-30T06:43:55.984ZTRACtest_utilization (oximeter-agent): ignoring request to update existing metric producer, since the endpoint information is the same as the existing collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
44882025-04-30T06:43:55.984ZTRACtest_utilization (oximeter-agent): ignoring request to update existing metric producer, since the endpoint information is the same as the existing collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = ac55d8d9-a298-4534-898b-beee1c9e4e52
44892025-04-30T06:43:55.984ZINFOtest_utilization (oximeter-agent): refreshed list of producers from Nexus collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 file = oximeter/collector/src/agent.rs:637 n_current_tasks = 2 n_pruned_tasks = 0
44902025-04-30T06:43:55.986ZDEBGtest_utilization (SEC): saga log event new_state = N053 started sec_id = ac55d8d9-a298-4534-898b-beee1c9e4e52
44912025-04-30T06:43:55.986ZDEBGtest_utilization (SecStore): recording saga event event_type = Started node_id = 53 saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c
44922025-04-30T06:43:55.990ZDEBGtest_utilization (SEC): saga log event new_state = N053 succeeded sec_id = ac55d8d9-a298-4534-898b-beee1c9e4e52
44932025-04-30T06:43:55.990ZDEBGtest_utilization (SecStore): recording saga event event_type = Succeeded(Object {"instance_id": String("d7bd2deb-ee27-4661-9e9a-524d734cd6ec"), "new_id": String("1db204ac-45f7-4301-b4bb-8b82e9fce5f5"), "saga_params": Object {"boundary_switches": Array [], "create_params": Object {"anti_affinity_groups": Array [], "auto_restart_policy": Null, "boot_disk": Null, "description": String("instance \\"utilization-test-instance\\""), "disks": Array [], "external_ips": Array [], "hostname": String("the-host"), "memory": Number(1073741824), "name": String("utilization-test-instance"), "ncpus": Number(4), "network_interfaces": Object {"type": String("default")}, "ssh_public_keys": Array [], "start": Bool(true), "user_data": String("I2Nsb3VkLWNvbmZpZwpzeXN0ZW1faW5mbzoKICBkZWZhdWx0X3VzZXI6CiAgICBuYW1lOiBveGlkZQ==")}, "project_id": String("473b39f9-8669-45b3-b3d7-793f7e10fc31"), "serialized_authn": Object {"kind": Object {"Authenticated": Array [Object {"actor": Object {"SiloUser": Object {"silo_id": String("001de000-5110-4000-8000-000000000000"), "silo_user_id": String("001de000-05e4-4000-8000-000000004007")}}}, Object {"mapped_fleet_roles": Object {}}]}}}, "which": Number(4)}) node_id = 53 saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c
44942025-04-30T06:43:55.993ZDEBGtest_utilization (SEC): saga log event new_state = N054 started sec_id = ac55d8d9-a298-4534-898b-beee1c9e4e52
44952025-04-30T06:43:55.993ZDEBGtest_utilization (SecStore): recording saga event event_type = Started node_id = 54 saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c
44962025-04-30T06:43:55.998ZDEBGtest_utilization (collection-task): sent timer-based collection request to the collection task collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
44972025-04-30T06:43:55.998ZDEBGtest_utilization (collection-task): collecting from producer collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
44982025-04-30T06:43:55.999ZDEBGtest_utilization (SEC): saga log event new_state = N054 succeeded sec_id = ac55d8d9-a298-4534-898b-beee1c9e4e52
44992025-04-30T06:43:55.999ZDEBGtest_utilization (SecStore): recording saga event event_type = Succeeded(Null) node_id = 54 saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c
45002025-04-30T06:43:56.010ZTRACtest_utilization (collection-task): sent collection request to producer collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
45012025-04-30T06:43:56.011ZDEBGtest_utilization (collection-task): collected results from producer collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 n_results = 1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
45022025-04-30T06:43:56.011ZTRACtest_utilization (collection-task): forwarded results to main collection loop collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
45032025-04-30T06:43:56.011ZTRACtest_utilization (collection-task): top of inner collection loop, waiting for next request collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
45042025-04-30T06:43:56.014ZDEBGtest_utilization (SEC): saga log event new_state = N055 started sec_id = ac55d8d9-a298-4534-898b-beee1c9e4e52
45052025-04-30T06:43:56.014ZDEBGtest_utilization (SecStore): recording saga event event_type = Started node_id = 55 saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c
45062025-04-30T06:43:56.061ZDEBGtest_utilization (SEC): saga log event new_state = N055 succeeded sec_id = ac55d8d9-a298-4534-898b-beee1c9e4e52
45072025-04-30T06:43:56.061ZDEBGtest_utilization (SecStore): recording saga event event_type = Succeeded(Null) node_id = 55 saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c
45082025-04-30T06:43:56.086ZDEBGtest_utilization (SEC): saga log event new_state = N056 started sec_id = ac55d8d9-a298-4534-898b-beee1c9e4e52
45092025-04-30T06:43:56.097ZDEBGtest_utilization (SecStore): recording saga event event_type = Started node_id = 56 saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c
45102025-04-30T06:43:56.106ZDEBGtest_utilization (SEC): saga log event new_state = N056 succeeded sec_id = ac55d8d9-a298-4534-898b-beee1c9e4e52
45112025-04-30T06:43:56.106ZDEBGtest_utilization (SecStore): recording saga event event_type = Succeeded(Null) node_id = 56 saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c
45122025-04-30T06:43:56.113ZDEBGtest_utilization (SEC): saga log event new_state = N057 started sec_id = ac55d8d9-a298-4534-898b-beee1c9e4e52
45132025-04-30T06:43:56.113ZDEBGtest_utilization (SecStore): recording saga event event_type = Started node_id = 57 saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c
45142025-04-30T06:43:56.137ZDEBGtest_utilization (SEC): saga log event new_state = N057 succeeded sec_id = ac55d8d9-a298-4534-898b-beee1c9e4e52
45152025-04-30T06:43:56.137ZDEBGtest_utilization (SecStore): recording saga event event_type = Succeeded(Object {"instance_id": String("d7bd2deb-ee27-4661-9e9a-524d734cd6ec"), "new_id": String("caf1be44-80eb-4e28-91d3-8f81c65992f5"), "saga_params": Object {"boundary_switches": Array [], "create_params": Object {"anti_affinity_groups": Array [], "auto_restart_policy": Null, "boot_disk": Null, "description": String("instance \\"utilization-test-instance\\""), "disks": Array [], "external_ips": Array [], "hostname": String("the-host"), "memory": Number(1073741824), "name": String("utilization-test-instance"), "ncpus": Number(4), "network_interfaces": Object {"type": String("default")}, "ssh_public_keys": Array [], "start": Bool(true), "user_data": String("I2Nsb3VkLWNvbmZpZwpzeXN0ZW1faW5mbzoKICBkZWZhdWx0X3VzZXI6CiAgICBuYW1lOiBveGlkZQ==")}, "project_id": String("473b39f9-8669-45b3-b3d7-793f7e10fc31"), "serialized_authn": Object {"kind": Object {"Authenticated": Array [Object {"actor": Object {"SiloUser": Object {"silo_id": String("001de000-5110-4000-8000-000000000000"), "silo_user_id": String("001de000-05e4-4000-8000-000000004007")}}}, Object {"mapped_fleet_roles": Object {}}]}}}, "which": Number(5)}) node_id = 57 saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c
45162025-04-30T06:43:56.147ZDEBGtest_utilization (SEC): saga log event new_state = N058 started sec_id = ac55d8d9-a298-4534-898b-beee1c9e4e52
45172025-04-30T06:43:56.148ZDEBGtest_utilization (SecStore): recording saga event event_type = Started node_id = 58 saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c
45182025-04-30T06:43:56.162ZDEBGtest_utilization (SEC): saga log event new_state = N058 succeeded sec_id = ac55d8d9-a298-4534-898b-beee1c9e4e52
45192025-04-30T06:43:56.162ZDEBGtest_utilization (SecStore): recording saga event event_type = Succeeded(Null) node_id = 58 saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c
45202025-04-30T06:43:56.167ZDEBGtest_utilization (SEC): saga log event new_state = N059 started sec_id = ac55d8d9-a298-4534-898b-beee1c9e4e52
45212025-04-30T06:43:56.167ZDEBGtest_utilization (SecStore): recording saga event event_type = Started node_id = 59 saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c
45222025-04-30T06:43:56.171ZDEBGtest_utilization (SEC): saga log event new_state = N059 succeeded sec_id = ac55d8d9-a298-4534-898b-beee1c9e4e52
45232025-04-30T06:43:56.171ZDEBGtest_utilization (SecStore): recording saga event event_type = Succeeded(Null) node_id = 59 saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c
45242025-04-30T06:43:56.199ZDEBGtest_utilization (SEC): saga log event new_state = N060 started sec_id = ac55d8d9-a298-4534-898b-beee1c9e4e52
45252025-04-30T06:43:56.199ZDEBGtest_utilization (SecStore): recording saga event event_type = Started node_id = 60 saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c
45262025-04-30T06:43:56.223ZDEBGtest_utilization (SEC): saga log event new_state = N060 succeeded sec_id = ac55d8d9-a298-4534-898b-beee1c9e4e52
45272025-04-30T06:43:56.223ZDEBGtest_utilization (SecStore): recording saga event event_type = Succeeded(Null) node_id = 60 saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c
45282025-04-30T06:43:56.231ZDEBGtest_utilization (SEC): saga log event new_state = N061 started sec_id = ac55d8d9-a298-4534-898b-beee1c9e4e52
45292025-04-30T06:43:56.231ZDEBGtest_utilization (SecStore): recording saga event event_type = Started node_id = 61 saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c
45302025-04-30T06:43:56.250ZDEBGtest_utilization (SEC): saga log event new_state = N061 succeeded sec_id = ac55d8d9-a298-4534-898b-beee1c9e4e52
45312025-04-30T06:43:56.250ZDEBGtest_utilization (SecStore): recording saga event event_type = Succeeded(Object {"instance_id": String("d7bd2deb-ee27-4661-9e9a-524d734cd6ec"), "new_id": String("c7469230-9dc6-409d-93da-9629008d5d16"), "saga_params": Object {"boundary_switches": Array [], "create_params": Object {"anti_affinity_groups": Array [], "auto_restart_policy": Null, "boot_disk": Null, "description": String("instance \\"utilization-test-instance\\""), "disks": Array [], "external_ips": Array [], "hostname": String("the-host"), "memory": Number(1073741824), "name": String("utilization-test-instance"), "ncpus": Number(4), "network_interfaces": Object {"type": String("default")}, "ssh_public_keys": Array [], "start": Bool(true), "user_data": String("I2Nsb3VkLWNvbmZpZwpzeXN0ZW1faW5mbzoKICBkZWZhdWx0X3VzZXI6CiAgICBuYW1lOiBveGlkZQ==")}, "project_id": String("473b39f9-8669-45b3-b3d7-793f7e10fc31"), "serialized_authn": Object {"kind": Object {"Authenticated": Array [Object {"actor": Object {"SiloUser": Object {"silo_id": String("001de000-5110-4000-8000-000000000000"), "silo_user_id": String("001de000-05e4-4000-8000-000000004007")}}}, Object {"mapped_fleet_roles": Object {}}]}}}, "which": Number(6)}) node_id = 61 saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c
45322025-04-30T06:43:56.271ZDEBGtest_utilization (SEC): saga log event new_state = N062 started sec_id = ac55d8d9-a298-4534-898b-beee1c9e4e52
45332025-04-30T06:43:56.271ZDEBGtest_utilization (SecStore): recording saga event event_type = Started node_id = 62 saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c
45342025-04-30T06:43:56.276ZDEBGtest_utilization (SEC): saga log event new_state = N062 succeeded sec_id = ac55d8d9-a298-4534-898b-beee1c9e4e52
45352025-04-30T06:43:56.276ZDEBGtest_utilization (SecStore): recording saga event event_type = Succeeded(Null) node_id = 62 saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c
45362025-04-30T06:43:56.281ZDEBGtest_utilization (SEC): saga log event new_state = N063 started sec_id = ac55d8d9-a298-4534-898b-beee1c9e4e52
45372025-04-30T06:43:56.281ZDEBGtest_utilization (SecStore): recording saga event event_type = Started node_id = 63 saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c
45382025-04-30T06:43:56.286ZDEBGtest_utilization (SEC): saga log event new_state = N063 succeeded sec_id = ac55d8d9-a298-4534-898b-beee1c9e4e52
45392025-04-30T06:43:56.286ZDEBGtest_utilization (SecStore): recording saga event event_type = Succeeded(Null) node_id = 63 saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c
45402025-04-30T06:43:56.295ZDEBGtest_utilization (SEC): saga log event new_state = N064 started sec_id = ac55d8d9-a298-4534-898b-beee1c9e4e52
45412025-04-30T06:43:56.295ZDEBGtest_utilization (SecStore): recording saga event event_type = Started node_id = 64 saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c
45422025-04-30T06:43:56.305ZDEBGtest_utilization (SEC): saga log event new_state = N064 succeeded sec_id = ac55d8d9-a298-4534-898b-beee1c9e4e52
45432025-04-30T06:43:56.305ZDEBGtest_utilization (SecStore): recording saga event event_type = Succeeded(Null) node_id = 64 saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c
45442025-04-30T06:43:56.340ZDEBGtest_utilization (SEC): saga log event new_state = N065 started sec_id = ac55d8d9-a298-4534-898b-beee1c9e4e52
45452025-04-30T06:43:56.341ZDEBGtest_utilization (SecStore): recording saga event event_type = Started node_id = 65 saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c
45462025-04-30T06:43:56.366ZDEBGtest_utilization (results-sink): inserting 1 samples into database collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
45472025-04-30T06:43:56.366ZDEBGtest_utilization (results-sink-cluster): inserting 1 samples into database collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
45482025-04-30T06:43:56.366ZDEBGtest_utilization (clickhouse-client): unrolling 1 total samples collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 4d1485f0-110e-4161-9a33-c4610d68be5d
45492025-04-30T06:43:56.367ZTRACtest_utilization (clickhouse-client): inserting data collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 4d1485f0-110e-4161-9a33-c4610d68be5d n_columns = 4 n_rows = 2 sql = INSERT INTO oximeter.fields_string FORMAT Native
45502025-04-30T06:43:56.383ZDEBGtest_utilization (clickhouse-client): inserted rows into table collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 4d1485f0-110e-4161-9a33-c4610d68be5d n_rows = 2 table_name = fields_string
45512025-04-30T06:43:56.383ZTRACtest_utilization (clickhouse-client): inserting data collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 4d1485f0-110e-4161-9a33-c4610d68be5d n_columns = 4 n_rows = 1 sql = INSERT INTO oximeter.measurements_i64 FORMAT Native
45522025-04-30T06:43:56.383ZDEBGtest_utilization (SEC): saga log event new_state = N065 succeeded sec_id = ac55d8d9-a298-4534-898b-beee1c9e4e52
45532025-04-30T06:43:56.383ZDEBGtest_utilization (SecStore): recording saga event event_type = Succeeded(Object {"instance_id": String("d7bd2deb-ee27-4661-9e9a-524d734cd6ec"), "new_id": String("1943d0ee-055e-4aa1-a7a8-403496b4dbcb"), "saga_params": Object {"boundary_switches": Array [], "create_params": Object {"anti_affinity_groups": Array [], "auto_restart_policy": Null, "boot_disk": Null, "description": String("instance \\"utilization-test-instance\\""), "disks": Array [], "external_ips": Array [], "hostname": String("the-host"), "memory": Number(1073741824), "name": String("utilization-test-instance"), "ncpus": Number(4), "network_interfaces": Object {"type": String("default")}, "ssh_public_keys": Array [], "start": Bool(true), "user_data": String("I2Nsb3VkLWNvbmZpZwpzeXN0ZW1faW5mbzoKICBkZWZhdWx0X3VzZXI6CiAgICBuYW1lOiBveGlkZQ==")}, "project_id": String("473b39f9-8669-45b3-b3d7-793f7e10fc31"), "serialized_authn": Object {"kind": Object {"Authenticated": Array [Object {"actor": Object {"SiloUser": Object {"silo_id": String("001de000-5110-4000-8000-000000000000"), "silo_user_id": String("001de000-05e4-4000-8000-000000004007")}}}, Object {"mapped_fleet_roles": Object {}}]}}}, "which": Number(7)}) node_id = 65 saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c
45542025-04-30T06:43:56.389ZDEBGtest_utilization (SEC): saga log event new_state = N066 started sec_id = ac55d8d9-a298-4534-898b-beee1c9e4e52
45552025-04-30T06:43:56.389ZDEBGtest_utilization (SecStore): recording saga event event_type = Started node_id = 66 saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c
45562025-04-30T06:43:56.392ZDEBGtest_utilization (clickhouse-client): inserted rows into table collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 4d1485f0-110e-4161-9a33-c4610d68be5d n_rows = 1 table_name = measurements_i64
45572025-04-30T06:43:56.392ZTRACtest_utilization (results-sink): successfully inserted samples collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
45582025-04-30T06:43:56.409ZDEBGtest_utilization (SEC): saga log event new_state = N066 succeeded sec_id = ac55d8d9-a298-4534-898b-beee1c9e4e52
45592025-04-30T06:43:56.409ZDEBGtest_utilization (SecStore): recording saga event event_type = Succeeded(Null) node_id = 66 saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c
45602025-04-30T06:43:56.413ZDEBGtest_utilization (SEC): saga log event new_state = N067 started sec_id = ac55d8d9-a298-4534-898b-beee1c9e4e52
45612025-04-30T06:43:56.413ZDEBGtest_utilization (SecStore): recording saga event event_type = Started node_id = 67 saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c
45622025-04-30T06:43:56.418ZDEBGtest_utilization (SEC): saga log event new_state = N067 succeeded sec_id = ac55d8d9-a298-4534-898b-beee1c9e4e52
45632025-04-30T06:43:56.418ZDEBGtest_utilization (SecStore): recording saga event event_type = Succeeded(Null) node_id = 67 saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c
45642025-04-30T06:43:56.441ZDEBGtest_utilization (SEC): saga log event new_state = N068 started sec_id = ac55d8d9-a298-4534-898b-beee1c9e4e52
45652025-04-30T06:43:56.441ZDEBGtest_utilization (SecStore): recording saga event event_type = Started node_id = 68 saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c
45662025-04-30T06:43:56.471ZWARNtest_utilization (results-sink-cluster): failed to insert some results into metric DB collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 error = Failed to check out connection to database: No backends found for this service file = oximeter/collector/src/results_sink.rs:92
45672025-04-30T06:43:56.471ZDEBGtest_utilization (SEC): saga log event new_state = N068 succeeded sec_id = ac55d8d9-a298-4534-898b-beee1c9e4e52
45682025-04-30T06:43:56.471ZDEBGtest_utilization (SecStore): recording saga event event_type = Succeeded(Null) node_id = 68 saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c
45692025-04-30T06:43:56.476ZDEBGtest_utilization (SEC): saga log event new_state = N069 started sec_id = ac55d8d9-a298-4534-898b-beee1c9e4e52
45702025-04-30T06:43:56.477ZDEBGtest_utilization (SecStore): recording saga event event_type = Started node_id = 69 saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c
45712025-04-30T06:43:56.479ZDEBGtest_utilization (SEC): saga log event new_state = N069 succeeded sec_id = ac55d8d9-a298-4534-898b-beee1c9e4e52
45722025-04-30T06:43:56.479ZDEBGtest_utilization (SecStore): recording saga event event_type = Succeeded(Object {"instance_id": String("d7bd2deb-ee27-4661-9e9a-524d734cd6ec"), "new_id": String("e9418c49-b114-4554-82a3-43f65092bf41"), "saga_params": Object {"boundary_switches": Array [], "create_params": Object {"anti_affinity_groups": Array [], "auto_restart_policy": Null, "boot_disk": Null, "description": String("instance \\"utilization-test-instance\\""), "disks": Array [], "external_ips": Array [], "hostname": String("the-host"), "memory": Number(1073741824), "name": String("utilization-test-instance"), "ncpus": Number(4), "network_interfaces": Object {"type": String("default")}, "ssh_public_keys": Array [], "start": Bool(true), "user_data": String("I2Nsb3VkLWNvbmZpZwpzeXN0ZW1faW5mbzoKICBkZWZhdWx0X3VzZXI6CiAgICBuYW1lOiBveGlkZQ==")}, "project_id": String("473b39f9-8669-45b3-b3d7-793f7e10fc31"), "serialized_authn": Object {"kind": Object {"Authenticated": Array [Object {"actor": Object {"SiloUser": Object {"silo_id": String("001de000-5110-4000-8000-000000000000"), "silo_user_id": String("001de000-05e4-4000-8000-000000004007")}}}, Object {"mapped_fleet_roles": Object {}}]}}}, "which": Number(8)}) node_id = 69 saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c
45732025-04-30T06:43:56.483ZDEBGtest_utilization (SEC): saga log event new_state = N070 started sec_id = ac55d8d9-a298-4534-898b-beee1c9e4e52
45742025-04-30T06:43:56.483ZDEBGtest_utilization (SecStore): recording saga event event_type = Started node_id = 70 saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c
45752025-04-30T06:43:56.502ZDEBGtest_utilization (SEC): saga log event new_state = N070 succeeded sec_id = ac55d8d9-a298-4534-898b-beee1c9e4e52
45762025-04-30T06:43:56.502ZDEBGtest_utilization (SecStore): recording saga event event_type = Succeeded(Null) node_id = 70 saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c
45772025-04-30T06:43:56.505ZDEBGtest_utilization (SEC): saga log event new_state = N071 started sec_id = ac55d8d9-a298-4534-898b-beee1c9e4e52
45782025-04-30T06:43:56.506ZDEBGtest_utilization (SecStore): recording saga event event_type = Started node_id = 71 saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c
45792025-04-30T06:43:56.541ZDEBGtest_utilization (SEC): saga log event new_state = N071 succeeded sec_id = ac55d8d9-a298-4534-898b-beee1c9e4e52
45802025-04-30T06:43:56.542ZDEBGtest_utilization (SecStore): recording saga event event_type = Succeeded(Null) node_id = 71 saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c
45812025-04-30T06:43:56.545ZDEBGtest_utilization (SEC): saga log event new_state = N072 started sec_id = ac55d8d9-a298-4534-898b-beee1c9e4e52
45822025-04-30T06:43:56.545ZDEBGtest_utilization (SecStore): recording saga event event_type = Started node_id = 72 saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c
45832025-04-30T06:43:56.548ZDEBGtest_utilization (SEC): saga log event new_state = N072 succeeded sec_id = ac55d8d9-a298-4534-898b-beee1c9e4e52
45842025-04-30T06:43:56.548ZDEBGtest_utilization (SecStore): recording saga event event_type = Succeeded(Null) node_id = 72 saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c
45852025-04-30T06:43:56.551ZDEBGtest_utilization (SEC): saga log event new_state = N073 started sec_id = ac55d8d9-a298-4534-898b-beee1c9e4e52
45862025-04-30T06:43:56.551ZDEBGtest_utilization (SecStore): recording saga event event_type = Started node_id = 73 saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c
45872025-04-30T06:43:56.554ZDEBGtest_utilization (SEC): saga log event new_state = N073 succeeded sec_id = ac55d8d9-a298-4534-898b-beee1c9e4e52
45882025-04-30T06:43:56.554ZDEBGtest_utilization (SecStore): recording saga event event_type = Succeeded(Object {"instance_id": String("d7bd2deb-ee27-4661-9e9a-524d734cd6ec"), "new_id": String("44dd706b-a60c-43ba-9fee-2f47969c2494"), "saga_params": Object {"boundary_switches": Array [], "create_params": Object {"anti_affinity_groups": Array [], "auto_restart_policy": Null, "boot_disk": Null, "description": String("instance \\"utilization-test-instance\\""), "disks": Array [], "external_ips": Array [], "hostname": String("the-host"), "memory": Number(1073741824), "name": String("utilization-test-instance"), "ncpus": Number(4), "network_interfaces": Object {"type": String("default")}, "ssh_public_keys": Array [], "start": Bool(true), "user_data": String("I2Nsb3VkLWNvbmZpZwpzeXN0ZW1faW5mbzoKICBkZWZhdWx0X3VzZXI6CiAgICBuYW1lOiBveGlkZQ==")}, "project_id": String("473b39f9-8669-45b3-b3d7-793f7e10fc31"), "serialized_authn": Object {"kind": Object {"Authenticated": Array [Object {"actor": Object {"SiloUser": Object {"silo_id": String("001de000-5110-4000-8000-000000000000"), "silo_user_id": String("001de000-05e4-4000-8000-000000004007")}}}, Object {"mapped_fleet_roles": Object {}}]}}}, "which": Number(9)}) node_id = 73 saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c
45892025-04-30T06:43:56.558ZDEBGtest_utilization (SEC): saga log event new_state = N074 started sec_id = ac55d8d9-a298-4534-898b-beee1c9e4e52
45902025-04-30T06:43:56.558ZDEBGtest_utilization (SecStore): recording saga event event_type = Started node_id = 74 saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c
45912025-04-30T06:43:56.560ZDEBGtest_utilization (SEC): saga log event new_state = N074 succeeded sec_id = ac55d8d9-a298-4534-898b-beee1c9e4e52
45922025-04-30T06:43:56.560ZDEBGtest_utilization (SecStore): recording saga event event_type = Succeeded(Null) node_id = 74 saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c
45932025-04-30T06:43:56.563ZDEBGtest_utilization (SEC): saga log event new_state = N075 started sec_id = ac55d8d9-a298-4534-898b-beee1c9e4e52
45942025-04-30T06:43:56.563ZDEBGtest_utilization (SecStore): recording saga event event_type = Started node_id = 75 saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c
45952025-04-30T06:43:56.571ZDEBGtest_utilization (SEC): saga log event new_state = N075 succeeded sec_id = ac55d8d9-a298-4534-898b-beee1c9e4e52
45962025-04-30T06:43:56.571ZDEBGtest_utilization (SecStore): recording saga event event_type = Succeeded(Null) node_id = 75 saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c
45972025-04-30T06:43:56.617ZDEBGtest_utilization (SEC): saga log event new_state = N076 started sec_id = ac55d8d9-a298-4534-898b-beee1c9e4e52
45982025-04-30T06:43:56.617ZDEBGtest_utilization (SecStore): recording saga event event_type = Started node_id = 76 saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c
45992025-04-30T06:43:56.623ZDEBGtest_utilization (SEC): saga log event new_state = N076 succeeded sec_id = ac55d8d9-a298-4534-898b-beee1c9e4e52
46002025-04-30T06:43:56.623ZDEBGtest_utilization (SecStore): recording saga event event_type = Succeeded(Null) node_id = 76 saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c
46012025-04-30T06:43:56.632ZDEBGtest_utilization (SEC): saga log event new_state = N077 started sec_id = ac55d8d9-a298-4534-898b-beee1c9e4e52
46022025-04-30T06:43:56.632ZDEBGtest_utilization (SecStore): recording saga event event_type = Started node_id = 77 saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c
46032025-04-30T06:43:56.655ZDEBGtest_utilization (SEC): saga log event new_state = N077 succeeded sec_id = ac55d8d9-a298-4534-898b-beee1c9e4e52
46042025-04-30T06:43:56.655ZDEBGtest_utilization (SecStore): recording saga event event_type = Succeeded(Object {"instance_id": String("d7bd2deb-ee27-4661-9e9a-524d734cd6ec"), "new_id": String("22abc147-674c-4b2d-aaaa-f1b4e56bec0d"), "saga_params": Object {"boundary_switches": Array [], "create_params": Object {"anti_affinity_groups": Array [], "auto_restart_policy": Null, "boot_disk": Null, "description": String("instance \\"utilization-test-instance\\""), "disks": Array [], "external_ips": Array [], "hostname": String("the-host"), "memory": Number(1073741824), "name": String("utilization-test-instance"), "ncpus": Number(4), "network_interfaces": Object {"type": String("default")}, "ssh_public_keys": Array [], "start": Bool(true), "user_data": String("I2Nsb3VkLWNvbmZpZwpzeXN0ZW1faW5mbzoKICBkZWZhdWx0X3VzZXI6CiAgICBuYW1lOiBveGlkZQ==")}, "project_id": String("473b39f9-8669-45b3-b3d7-793f7e10fc31"), "serialized_authn": Object {"kind": Object {"Authenticated": Array [Object {"actor": Object {"SiloUser": Object {"silo_id": String("001de000-5110-4000-8000-000000000000"), "silo_user_id": String("001de000-05e4-4000-8000-000000004007")}}}, Object {"mapped_fleet_roles": Object {}}]}}}, "which": Number(10)}) node_id = 77 saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c
46052025-04-30T06:43:56.693ZDEBGtest_utilization (SEC): saga log event new_state = N078 started sec_id = ac55d8d9-a298-4534-898b-beee1c9e4e52
46062025-04-30T06:43:56.693ZDEBGtest_utilization (SecStore): recording saga event event_type = Started node_id = 78 saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c
46072025-04-30T06:43:56.725ZDEBGtest_utilization (SEC): saga log event new_state = N078 succeeded sec_id = ac55d8d9-a298-4534-898b-beee1c9e4e52
46082025-04-30T06:43:56.725ZDEBGtest_utilization (SecStore): recording saga event event_type = Succeeded(Null) node_id = 78 saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c
46092025-04-30T06:43:56.730ZDEBGtest_utilization (SEC): saga log event new_state = N079 started sec_id = ac55d8d9-a298-4534-898b-beee1c9e4e52
46102025-04-30T06:43:56.730ZDEBGtest_utilization (SecStore): recording saga event event_type = Started node_id = 79 saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c
46112025-04-30T06:43:56.735ZDEBGtest_utilization (SEC): saga log event new_state = N079 succeeded sec_id = ac55d8d9-a298-4534-898b-beee1c9e4e52
46122025-04-30T06:43:56.735ZDEBGtest_utilization (SecStore): recording saga event event_type = Succeeded(Null) node_id = 79 saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c
46132025-04-30T06:43:56.798ZDEBGtest_utilization (SEC): saga log event new_state = N080 started sec_id = ac55d8d9-a298-4534-898b-beee1c9e4e52
46142025-04-30T06:43:56.798ZDEBGtest_utilization (SecStore): recording saga event event_type = Started node_id = 80 saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c
46152025-04-30T06:43:56.804ZDEBGtest_utilization (SEC): saga log event new_state = N080 succeeded sec_id = ac55d8d9-a298-4534-898b-beee1c9e4e52
46162025-04-30T06:43:56.804ZDEBGtest_utilization (SecStore): recording saga event event_type = Succeeded(Null) node_id = 80 saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c
46172025-04-30T06:43:56.807ZDEBGtest_utilization (SEC): saga log event new_state = N081 started sec_id = ac55d8d9-a298-4534-898b-beee1c9e4e52
46182025-04-30T06:43:56.807ZDEBGtest_utilization (SecStore): recording saga event event_type = Started node_id = 81 saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c
46192025-04-30T06:43:56.810ZDEBGtest_utilization (SEC): saga log event new_state = N081 succeeded sec_id = ac55d8d9-a298-4534-898b-beee1c9e4e52
46202025-04-30T06:43:56.810ZDEBGtest_utilization (SecStore): recording saga event event_type = Succeeded(Object {"instance_id": String("d7bd2deb-ee27-4661-9e9a-524d734cd6ec"), "new_id": String("90899943-1eb7-4a82-93b8-3ca4311fc81e"), "saga_params": Object {"boundary_switches": Array [], "create_params": Object {"anti_affinity_groups": Array [], "auto_restart_policy": Null, "boot_disk": Null, "description": String("instance \\"utilization-test-instance\\""), "disks": Array [], "external_ips": Array [], "hostname": String("the-host"), "memory": Number(1073741824), "name": String("utilization-test-instance"), "ncpus": Number(4), "network_interfaces": Object {"type": String("default")}, "ssh_public_keys": Array [], "start": Bool(true), "user_data": String("I2Nsb3VkLWNvbmZpZwpzeXN0ZW1faW5mbzoKICBkZWZhdWx0X3VzZXI6CiAgICBuYW1lOiBveGlkZQ==")}, "project_id": String("473b39f9-8669-45b3-b3d7-793f7e10fc31"), "serialized_authn": Object {"kind": Object {"Authenticated": Array [Object {"actor": Object {"SiloUser": Object {"silo_id": String("001de000-5110-4000-8000-000000000000"), "silo_user_id": String("001de000-05e4-4000-8000-000000004007")}}}, Object {"mapped_fleet_roles": Object {}}]}}}, "which": Number(11)}) node_id = 81 saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c
46212025-04-30T06:43:56.813ZDEBGtest_utilization (SEC): saga log event new_state = N082 started sec_id = ac55d8d9-a298-4534-898b-beee1c9e4e52
46222025-04-30T06:43:56.813ZDEBGtest_utilization (SecStore): recording saga event event_type = Started node_id = 82 saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c
46232025-04-30T06:43:56.823ZDEBGtest_utilization (SEC): saga log event new_state = N082 succeeded sec_id = ac55d8d9-a298-4534-898b-beee1c9e4e52
46242025-04-30T06:43:56.823ZDEBGtest_utilization (SecStore): recording saga event event_type = Succeeded(Null) node_id = 82 saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c
46252025-04-30T06:43:56.827ZDEBGtest_utilization (SEC): saga log event new_state = N083 started sec_id = ac55d8d9-a298-4534-898b-beee1c9e4e52
46262025-04-30T06:43:56.827ZDEBGtest_utilization (SecStore): recording saga event event_type = Started node_id = 83 saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c
46272025-04-30T06:43:56.830ZDEBGtest_utilization (SEC): saga log event new_state = N083 succeeded sec_id = ac55d8d9-a298-4534-898b-beee1c9e4e52
46282025-04-30T06:43:56.830ZDEBGtest_utilization (SecStore): recording saga event event_type = Succeeded(Null) node_id = 83 saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c
46292025-04-30T06:43:56.834ZDEBGtest_utilization (SEC): saga log event new_state = N084 started sec_id = ac55d8d9-a298-4534-898b-beee1c9e4e52
46302025-04-30T06:43:56.834ZDEBGtest_utilization (SecStore): recording saga event event_type = Started node_id = 84 saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c
46312025-04-30T06:43:56.837ZDEBGtest_utilization (SEC): saga log event new_state = N084 succeeded sec_id = ac55d8d9-a298-4534-898b-beee1c9e4e52
46322025-04-30T06:43:56.837ZDEBGtest_utilization (SecStore): recording saga event event_type = Succeeded(Null) node_id = 84 saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c
46332025-04-30T06:43:56.840ZDEBGtest_utilization (SEC): saga log event new_state = N085 started sec_id = ac55d8d9-a298-4534-898b-beee1c9e4e52
46342025-04-30T06:43:56.840ZDEBGtest_utilization (SecStore): recording saga event event_type = Started node_id = 85 saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c
46352025-04-30T06:43:56.843ZDEBGtest_utilization (SEC): saga log event new_state = N085 succeeded sec_id = ac55d8d9-a298-4534-898b-beee1c9e4e52
46362025-04-30T06:43:56.843ZDEBGtest_utilization (SecStore): recording saga event event_type = Succeeded(Object {"instance_id": String("d7bd2deb-ee27-4661-9e9a-524d734cd6ec"), "new_id": String("6979ef71-cbe0-4065-8900-ae9ef8686f28"), "saga_params": Object {"boundary_switches": Array [], "create_params": Object {"anti_affinity_groups": Array [], "auto_restart_policy": Null, "boot_disk": Null, "description": String("instance \\"utilization-test-instance\\""), "disks": Array [], "external_ips": Array [], "hostname": String("the-host"), "memory": Number(1073741824), "name": String("utilization-test-instance"), "ncpus": Number(4), "network_interfaces": Object {"type": String("default")}, "ssh_public_keys": Array [], "start": Bool(true), "user_data": String("I2Nsb3VkLWNvbmZpZwpzeXN0ZW1faW5mbzoKICBkZWZhdWx0X3VzZXI6CiAgICBuYW1lOiBveGlkZQ==")}, "project_id": String("473b39f9-8669-45b3-b3d7-793f7e10fc31"), "serialized_authn": Object {"kind": Object {"Authenticated": Array [Object {"actor": Object {"SiloUser": Object {"silo_id": String("001de000-5110-4000-8000-000000000000"), "silo_user_id": String("001de000-05e4-4000-8000-000000004007")}}}, Object {"mapped_fleet_roles": Object {}}]}}}, "which": Number(12)}) node_id = 85 saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c
46372025-04-30T06:43:56.846ZDEBGtest_utilization (SEC): saga log event new_state = N086 started sec_id = ac55d8d9-a298-4534-898b-beee1c9e4e52
46382025-04-30T06:43:56.846ZDEBGtest_utilization (SecStore): recording saga event event_type = Started node_id = 86 saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c
46392025-04-30T06:43:56.849ZDEBGtest_utilization (SEC): saga log event new_state = N086 succeeded sec_id = ac55d8d9-a298-4534-898b-beee1c9e4e52
46402025-04-30T06:43:56.849ZDEBGtest_utilization (SecStore): recording saga event event_type = Succeeded(Null) node_id = 86 saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c
46412025-04-30T06:43:56.852ZDEBGtest_utilization (SEC): saga log event new_state = N087 started sec_id = ac55d8d9-a298-4534-898b-beee1c9e4e52
46422025-04-30T06:43:56.852ZDEBGtest_utilization (SecStore): recording saga event event_type = Started node_id = 87 saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c
46432025-04-30T06:43:56.863ZDEBGtest_utilization (SEC): saga log event new_state = N087 succeeded sec_id = ac55d8d9-a298-4534-898b-beee1c9e4e52
46442025-04-30T06:43:56.863ZDEBGtest_utilization (SecStore): recording saga event event_type = Succeeded(Null) node_id = 87 saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c
46452025-04-30T06:43:56.867ZDEBGtest_utilization (SEC): saga log event new_state = N088 started sec_id = ac55d8d9-a298-4534-898b-beee1c9e4e52
46462025-04-30T06:43:56.867ZDEBGtest_utilization (SecStore): recording saga event event_type = Started node_id = 88 saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c
46472025-04-30T06:43:56.872ZDEBGtest_utilization (SEC): saga log event new_state = N088 succeeded sec_id = ac55d8d9-a298-4534-898b-beee1c9e4e52
46482025-04-30T06:43:56.872ZDEBGtest_utilization (SecStore): recording saga event event_type = Succeeded(Null) node_id = 88 saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c
46492025-04-30T06:43:56.888ZDEBGtest_utilization (SEC): saga log event new_state = N089 started sec_id = ac55d8d9-a298-4534-898b-beee1c9e4e52
46502025-04-30T06:43:56.888ZDEBGtest_utilization (SecStore): recording saga event event_type = Started node_id = 89 saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c
46512025-04-30T06:43:56.891ZDEBGtest_utilization (SEC): saga log event new_state = N089 succeeded sec_id = ac55d8d9-a298-4534-898b-beee1c9e4e52
46522025-04-30T06:43:56.891ZDEBGtest_utilization (SecStore): recording saga event event_type = Succeeded(Object {"instance_id": String("d7bd2deb-ee27-4661-9e9a-524d734cd6ec"), "new_id": String("5e5d8ca1-846b-484c-a5a7-5cc20d5140d3"), "saga_params": Object {"boundary_switches": Array [], "create_params": Object {"anti_affinity_groups": Array [], "auto_restart_policy": Null, "boot_disk": Null, "description": String("instance \\"utilization-test-instance\\""), "disks": Array [], "external_ips": Array [], "hostname": String("the-host"), "memory": Number(1073741824), "name": String("utilization-test-instance"), "ncpus": Number(4), "network_interfaces": Object {"type": String("default")}, "ssh_public_keys": Array [], "start": Bool(true), "user_data": String("I2Nsb3VkLWNvbmZpZwpzeXN0ZW1faW5mbzoKICBkZWZhdWx0X3VzZXI6CiAgICBuYW1lOiBveGlkZQ==")}, "project_id": String("473b39f9-8669-45b3-b3d7-793f7e10fc31"), "serialized_authn": Object {"kind": Object {"Authenticated": Array [Object {"actor": Object {"SiloUser": Object {"silo_id": String("001de000-5110-4000-8000-000000000000"), "silo_user_id": String("001de000-05e4-4000-8000-000000004007")}}}, Object {"mapped_fleet_roles": Object {}}]}}}, "which": Number(13)}) node_id = 89 saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c
46532025-04-30T06:43:56.899ZDEBGtest_utilization (SEC): saga log event new_state = N090 started sec_id = ac55d8d9-a298-4534-898b-beee1c9e4e52
46542025-04-30T06:43:56.899ZDEBGtest_utilization (SecStore): recording saga event event_type = Started node_id = 90 saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c
46552025-04-30T06:43:56.903ZDEBGtest_utilization (SEC): saga log event new_state = N090 succeeded sec_id = ac55d8d9-a298-4534-898b-beee1c9e4e52
46562025-04-30T06:43:56.903ZDEBGtest_utilization (SecStore): recording saga event event_type = Succeeded(Null) node_id = 90 saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c
46572025-04-30T06:43:56.906ZDEBGtest_utilization (SEC): saga log event new_state = N091 started sec_id = ac55d8d9-a298-4534-898b-beee1c9e4e52
46582025-04-30T06:43:56.906ZDEBGtest_utilization (SecStore): recording saga event event_type = Started node_id = 91 saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c
46592025-04-30T06:43:56.911ZDEBGtest_utilization (SEC): saga log event new_state = N091 succeeded sec_id = ac55d8d9-a298-4534-898b-beee1c9e4e52
46602025-04-30T06:43:56.911ZDEBGtest_utilization (SecStore): recording saga event event_type = Succeeded(Null) node_id = 91 saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c
46612025-04-30T06:43:56.916ZDEBGtest_utilization (SEC): saga log event new_state = N092 started sec_id = ac55d8d9-a298-4534-898b-beee1c9e4e52
46622025-04-30T06:43:56.916ZDEBGtest_utilization (SecStore): recording saga event event_type = Started node_id = 92 saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c
46632025-04-30T06:43:56.919ZDEBGtest_utilization (SEC): saga log event new_state = N092 succeeded sec_id = ac55d8d9-a298-4534-898b-beee1c9e4e52
46642025-04-30T06:43:56.919ZDEBGtest_utilization (SecStore): recording saga event event_type = Succeeded(Null) node_id = 92 saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c
46652025-04-30T06:43:56.922ZDEBGtest_utilization (SEC): saga log event new_state = N093 started sec_id = ac55d8d9-a298-4534-898b-beee1c9e4e52
46662025-04-30T06:43:56.922ZDEBGtest_utilization (SecStore): recording saga event event_type = Started node_id = 93 saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c
46672025-04-30T06:43:56.926ZDEBGtest_utilization (SEC): saga log event new_state = N093 succeeded sec_id = ac55d8d9-a298-4534-898b-beee1c9e4e52
46682025-04-30T06:43:56.927ZDEBGtest_utilization (SecStore): recording saga event event_type = Succeeded(Object {"instance_id": String("d7bd2deb-ee27-4661-9e9a-524d734cd6ec"), "new_id": String("f0b5c727-87b5-41de-aede-b1bc7b951264"), "saga_params": Object {"boundary_switches": Array [], "create_params": Object {"anti_affinity_groups": Array [], "auto_restart_policy": Null, "boot_disk": Null, "description": String("instance \\"utilization-test-instance\\""), "disks": Array [], "external_ips": Array [], "hostname": String("the-host"), "memory": Number(1073741824), "name": String("utilization-test-instance"), "ncpus": Number(4), "network_interfaces": Object {"type": String("default")}, "ssh_public_keys": Array [], "start": Bool(true), "user_data": String("I2Nsb3VkLWNvbmZpZwpzeXN0ZW1faW5mbzoKICBkZWZhdWx0X3VzZXI6CiAgICBuYW1lOiBveGlkZQ==")}, "project_id": String("473b39f9-8669-45b3-b3d7-793f7e10fc31"), "serialized_authn": Object {"kind": Object {"Authenticated": Array [Object {"actor": Object {"SiloUser": Object {"silo_id": String("001de000-5110-4000-8000-000000000000"), "silo_user_id": String("001de000-05e4-4000-8000-000000004007")}}}, Object {"mapped_fleet_roles": Object {}}]}}}, "which": Number(14)}) node_id = 93 saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c
46692025-04-30T06:43:56.935ZDEBGtest_utilization (SEC): saga log event new_state = N094 started sec_id = ac55d8d9-a298-4534-898b-beee1c9e4e52
46702025-04-30T06:43:56.935ZDEBGtest_utilization (SecStore): recording saga event event_type = Started node_id = 94 saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c
46712025-04-30T06:43:56.954ZDEBGtest_utilization (SEC): saga log event new_state = N094 succeeded sec_id = ac55d8d9-a298-4534-898b-beee1c9e4e52
46722025-04-30T06:43:56.954ZDEBGtest_utilization (SecStore): recording saga event event_type = Succeeded(Null) node_id = 94 saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c
46732025-04-30T06:43:56.965ZDEBGtest_utilization (SEC): saga log event new_state = N095 started sec_id = ac55d8d9-a298-4534-898b-beee1c9e4e52
46742025-04-30T06:43:56.965ZDEBGtest_utilization (SecStore): recording saga event event_type = Started node_id = 95 saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c
46752025-04-30T06:43:56.971ZDEBGtest_utilization (SEC): saga log event new_state = N095 succeeded sec_id = ac55d8d9-a298-4534-898b-beee1c9e4e52
46762025-04-30T06:43:56.975ZDEBGtest_utilization (SecStore): recording saga event event_type = Succeeded(Null) node_id = 95 saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c
46772025-04-30T06:43:56.980ZDEBGtest_utilization (SEC): saga log event new_state = N096 started sec_id = ac55d8d9-a298-4534-898b-beee1c9e4e52
46782025-04-30T06:43:56.981ZDEBGtest_utilization (SecStore): recording saga event event_type = Started node_id = 96 saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c
46792025-04-30T06:43:57.004ZDEBGtest_utilization (collection-task): sent timer-based collection request to the collection task collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
46802025-04-30T06:43:57.004ZDEBGtest_utilization (collection-task): collecting from producer collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
46812025-04-30T06:43:57.004ZDEBGtest_utilization (SEC): saga log event new_state = N096 succeeded sec_id = ac55d8d9-a298-4534-898b-beee1c9e4e52
46822025-04-30T06:43:57.004ZDEBGtest_utilization (SecStore): recording saga event event_type = Succeeded(Null) node_id = 96 saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c
46832025-04-30T06:43:57.007ZTRACtest_utilization (collection-task): sent collection request to producer collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
46842025-04-30T06:43:57.008ZDEBGtest_utilization (collection-task): collected results from producer collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 n_results = 1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
46852025-04-30T06:43:57.008ZTRACtest_utilization (collection-task): forwarded results to main collection loop collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
46862025-04-30T06:43:57.008ZTRACtest_utilization (collection-task): top of inner collection loop, waiting for next request collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
46872025-04-30T06:43:57.036ZDEBGtest_utilization (SEC): saga log event new_state = N097 started sec_id = ac55d8d9-a298-4534-898b-beee1c9e4e52
46882025-04-30T06:43:57.036ZDEBGtest_utilization (SecStore): recording saga event event_type = Started node_id = 97 saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c
46892025-04-30T06:43:57.053ZDEBGtest_utilization (SEC): saga log event new_state = N097 succeeded sec_id = ac55d8d9-a298-4534-898b-beee1c9e4e52
46902025-04-30T06:43:57.053ZDEBGtest_utilization (SecStore): recording saga event event_type = Succeeded(Object {"instance_id": String("d7bd2deb-ee27-4661-9e9a-524d734cd6ec"), "new_id": String("1c2a0f63-abc4-4eef-b4b2-335cad832ba3"), "saga_params": Object {"boundary_switches": Array [], "create_params": Object {"anti_affinity_groups": Array [], "auto_restart_policy": Null, "boot_disk": Null, "description": String("instance \\"utilization-test-instance\\""), "disks": Array [], "external_ips": Array [], "hostname": String("the-host"), "memory": Number(1073741824), "name": String("utilization-test-instance"), "ncpus": Number(4), "network_interfaces": Object {"type": String("default")}, "ssh_public_keys": Array [], "start": Bool(true), "user_data": String("I2Nsb3VkLWNvbmZpZwpzeXN0ZW1faW5mbzoKICBkZWZhdWx0X3VzZXI6CiAgICBuYW1lOiBveGlkZQ==")}, "project_id": String("473b39f9-8669-45b3-b3d7-793f7e10fc31"), "serialized_authn": Object {"kind": Object {"Authenticated": Array [Object {"actor": Object {"SiloUser": Object {"silo_id": String("001de000-5110-4000-8000-000000000000"), "silo_user_id": String("001de000-05e4-4000-8000-000000004007")}}}, Object {"mapped_fleet_roles": Object {}}]}}}, "which": Number(15)}) node_id = 97 saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c
46912025-04-30T06:43:57.074ZDEBGtest_utilization (SEC): saga log event new_state = N098 started sec_id = ac55d8d9-a298-4534-898b-beee1c9e4e52
46922025-04-30T06:43:57.078ZDEBGtest_utilization (SecStore): recording saga event event_type = Started node_id = 98 saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c
46932025-04-30T06:43:57.086ZDEBGtest_utilization (SEC): saga log event new_state = N098 succeeded sec_id = ac55d8d9-a298-4534-898b-beee1c9e4e52
46942025-04-30T06:43:57.087ZDEBGtest_utilization (SecStore): recording saga event event_type = Succeeded(Null) node_id = 98 saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c
46952025-04-30T06:43:57.091ZDEBGtest_utilization (SEC): saga log event new_state = N099 started sec_id = ac55d8d9-a298-4534-898b-beee1c9e4e52
46962025-04-30T06:43:57.092ZDEBGtest_utilization (SecStore): recording saga event event_type = Started node_id = 99 saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c
46972025-04-30T06:43:57.103ZDEBGtest_utilization (SEC): saga log event new_state = N099 succeeded sec_id = ac55d8d9-a298-4534-898b-beee1c9e4e52
46982025-04-30T06:43:57.103ZDEBGtest_utilization (SecStore): recording saga event event_type = Succeeded(Null) node_id = 99 saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c
46992025-04-30T06:43:57.107ZDEBGtest_utilization (SEC): saga log event new_state = N100 started sec_id = ac55d8d9-a298-4534-898b-beee1c9e4e52
47002025-04-30T06:43:57.107ZDEBGtest_utilization (SecStore): recording saga event event_type = Started node_id = 100 saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c
47012025-04-30T06:43:57.196ZDEBGtest_utilization (SEC): saga log event new_state = N100 succeeded sec_id = ac55d8d9-a298-4534-898b-beee1c9e4e52
47022025-04-30T06:43:57.196ZDEBGtest_utilization (SecStore): recording saga event event_type = Succeeded(Null) node_id = 100 saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c
47032025-04-30T06:43:57.218ZDEBGtest_utilization (SEC): saga log event new_state = N101 started sec_id = ac55d8d9-a298-4534-898b-beee1c9e4e52
47042025-04-30T06:43:57.218ZDEBGtest_utilization (SecStore): recording saga event event_type = Started node_id = 101 saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c
47052025-04-30T06:43:57.225ZDEBGtest_utilization (SEC): saga log event new_state = N101 succeeded sec_id = ac55d8d9-a298-4534-898b-beee1c9e4e52
47062025-04-30T06:43:57.225ZDEBGtest_utilization (SecStore): recording saga event event_type = Succeeded(Object {"instance_id": String("d7bd2deb-ee27-4661-9e9a-524d734cd6ec"), "new_id": String("f25005d7-f45c-4620-a7ed-94d2eda156e0"), "saga_params": Object {"boundary_switches": Array [], "create_params": Object {"anti_affinity_groups": Array [], "auto_restart_policy": Null, "boot_disk": Null, "description": String("instance \\"utilization-test-instance\\""), "disks": Array [], "external_ips": Array [], "hostname": String("the-host"), "memory": Number(1073741824), "name": String("utilization-test-instance"), "ncpus": Number(4), "network_interfaces": Object {"type": String("default")}, "ssh_public_keys": Array [], "start": Bool(true), "user_data": String("I2Nsb3VkLWNvbmZpZwpzeXN0ZW1faW5mbzoKICBkZWZhdWx0X3VzZXI6CiAgICBuYW1lOiBveGlkZQ==")}, "project_id": String("473b39f9-8669-45b3-b3d7-793f7e10fc31"), "serialized_authn": Object {"kind": Object {"Authenticated": Array [Object {"actor": Object {"SiloUser": Object {"silo_id": String("001de000-5110-4000-8000-000000000000"), "silo_user_id": String("001de000-05e4-4000-8000-000000004007")}}}, Object {"mapped_fleet_roles": Object {}}]}}}, "which": Number(16)}) node_id = 101 saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c
47072025-04-30T06:43:57.244ZDEBGtest_utilization (SEC): saga log event new_state = N102 started sec_id = ac55d8d9-a298-4534-898b-beee1c9e4e52
47082025-04-30T06:43:57.246ZDEBGtest_utilization (SecStore): recording saga event event_type = Started node_id = 102 saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c
47092025-04-30T06:43:57.252ZDEBGtest_utilization (SEC): saga log event new_state = N102 succeeded sec_id = ac55d8d9-a298-4534-898b-beee1c9e4e52
47102025-04-30T06:43:57.257ZDEBGtest_utilization (SecStore): recording saga event event_type = Succeeded(Null) node_id = 102 saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c
47112025-04-30T06:43:57.272ZDEBGtest_utilization (SEC): saga log event new_state = N103 started sec_id = ac55d8d9-a298-4534-898b-beee1c9e4e52
47122025-04-30T06:43:57.273ZDEBGtest_utilization (SecStore): recording saga event event_type = Started node_id = 103 saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c
47132025-04-30T06:43:57.277ZDEBGtest_utilization (SEC): saga log event new_state = N103 succeeded sec_id = ac55d8d9-a298-4534-898b-beee1c9e4e52
47142025-04-30T06:43:57.278ZDEBGtest_utilization (SecStore): recording saga event event_type = Succeeded(Null) node_id = 103 saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c
47152025-04-30T06:43:57.282ZDEBGtest_utilization (SEC): saga log event new_state = N104 started sec_id = ac55d8d9-a298-4534-898b-beee1c9e4e52
47162025-04-30T06:43:57.282ZDEBGtest_utilization (SecStore): recording saga event event_type = Started node_id = 104 saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c
47172025-04-30T06:43:57.286ZDEBGtest_utilization (SEC): saga log event new_state = N104 succeeded sec_id = ac55d8d9-a298-4534-898b-beee1c9e4e52
47182025-04-30T06:43:57.286ZDEBGtest_utilization (SecStore): recording saga event event_type = Succeeded(Null) node_id = 104 saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c
47192025-04-30T06:43:57.300ZDEBGtest_utilization (SEC): saga log event new_state = N105 started sec_id = ac55d8d9-a298-4534-898b-beee1c9e4e52
47202025-04-30T06:43:57.301ZDEBGtest_utilization (SecStore): recording saga event event_type = Started node_id = 105 saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c
47212025-04-30T06:43:57.304ZDEBGtest_utilization (SEC): saga log event new_state = N105 succeeded sec_id = ac55d8d9-a298-4534-898b-beee1c9e4e52
47222025-04-30T06:43:57.305ZDEBGtest_utilization (SecStore): recording saga event event_type = Succeeded(Object {"instance_id": String("d7bd2deb-ee27-4661-9e9a-524d734cd6ec"), "new_id": String("7340d976-adcc-4ae0-b83b-0dd30bc87c58"), "saga_params": Object {"boundary_switches": Array [], "create_params": Object {"anti_affinity_groups": Array [], "auto_restart_policy": Null, "boot_disk": Null, "description": String("instance \\"utilization-test-instance\\""), "disks": Array [], "external_ips": Array [], "hostname": String("the-host"), "memory": Number(1073741824), "name": String("utilization-test-instance"), "ncpus": Number(4), "network_interfaces": Object {"type": String("default")}, "ssh_public_keys": Array [], "start": Bool(true), "user_data": String("I2Nsb3VkLWNvbmZpZwpzeXN0ZW1faW5mbzoKICBkZWZhdWx0X3VzZXI6CiAgICBuYW1lOiBveGlkZQ==")}, "project_id": String("473b39f9-8669-45b3-b3d7-793f7e10fc31"), "serialized_authn": Object {"kind": Object {"Authenticated": Array [Object {"actor": Object {"SiloUser": Object {"silo_id": String("001de000-5110-4000-8000-000000000000"), "silo_user_id": String("001de000-05e4-4000-8000-000000004007")}}}, Object {"mapped_fleet_roles": Object {}}]}}}, "which": Number(17)}) node_id = 105 saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c
47232025-04-30T06:43:57.311ZDEBGtest_utilization (SEC): saga log event new_state = N106 started sec_id = ac55d8d9-a298-4534-898b-beee1c9e4e52
47242025-04-30T06:43:57.312ZDEBGtest_utilization (SecStore): recording saga event event_type = Started node_id = 106 saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c
47252025-04-30T06:43:57.335ZDEBGtest_utilization (SEC): saga log event new_state = N106 succeeded sec_id = ac55d8d9-a298-4534-898b-beee1c9e4e52
47262025-04-30T06:43:57.335ZDEBGtest_utilization (SecStore): recording saga event event_type = Succeeded(Null) node_id = 106 saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c
47272025-04-30T06:43:57.356ZDEBGtest_utilization (SEC): saga log event new_state = N107 started sec_id = ac55d8d9-a298-4534-898b-beee1c9e4e52
47282025-04-30T06:43:57.356ZDEBGtest_utilization (SecStore): recording saga event event_type = Started node_id = 107 saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c
47292025-04-30T06:43:57.357ZDEBGtest_utilization (results-sink): inserting 1 samples into database collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
47302025-04-30T06:43:57.357ZDEBGtest_utilization (results-sink-cluster): inserting 1 samples into database collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
47312025-04-30T06:43:57.357ZDEBGtest_utilization (clickhouse-client): unrolling 1 total samples collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 4d1485f0-110e-4161-9a33-c4610d68be5d
47322025-04-30T06:43:57.358ZTRACtest_utilization (clickhouse-client): inserting data collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 4d1485f0-110e-4161-9a33-c4610d68be5d n_columns = 4 n_rows = 2 sql = INSERT INTO oximeter.fields_string FORMAT Native
47332025-04-30T06:43:57.362ZDEBGtest_utilization (SEC): saga log event new_state = N107 succeeded sec_id = ac55d8d9-a298-4534-898b-beee1c9e4e52
47342025-04-30T06:43:57.362ZDEBGtest_utilization (SecStore): recording saga event event_type = Succeeded(Null) node_id = 107 saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c
47352025-04-30T06:43:57.365ZDEBGtest_utilization (clickhouse-client): inserted rows into table collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 4d1485f0-110e-4161-9a33-c4610d68be5d n_rows = 2 table_name = fields_string
47362025-04-30T06:43:57.366ZTRACtest_utilization (clickhouse-client): inserting data collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 4d1485f0-110e-4161-9a33-c4610d68be5d n_columns = 4 n_rows = 1 sql = INSERT INTO oximeter.measurements_i64 FORMAT Native
47372025-04-30T06:43:57.366ZDEBGtest_utilization (SEC): saga log event new_state = N108 started sec_id = ac55d8d9-a298-4534-898b-beee1c9e4e52
47382025-04-30T06:43:57.366ZDEBGtest_utilization (SecStore): recording saga event event_type = Started node_id = 108 saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c
47392025-04-30T06:43:57.370ZDEBGtest_utilization (clickhouse-client): inserted rows into table collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 4d1485f0-110e-4161-9a33-c4610d68be5d n_rows = 1 table_name = measurements_i64
47402025-04-30T06:43:57.370ZTRACtest_utilization (results-sink): successfully inserted samples collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
47412025-04-30T06:43:57.370ZDEBGtest_utilization (SEC): saga log event new_state = N108 succeeded sec_id = ac55d8d9-a298-4534-898b-beee1c9e4e52
47422025-04-30T06:43:57.370ZDEBGtest_utilization (SecStore): recording saga event event_type = Succeeded(Null) node_id = 108 saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c
47432025-04-30T06:43:57.374ZDEBGtest_utilization (SEC): saga log event new_state = N109 started sec_id = ac55d8d9-a298-4534-898b-beee1c9e4e52
47442025-04-30T06:43:57.374ZDEBGtest_utilization (SecStore): recording saga event event_type = Started node_id = 109 saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c
47452025-04-30T06:43:57.379ZDEBGtest_utilization (SEC): saga log event new_state = N109 succeeded sec_id = ac55d8d9-a298-4534-898b-beee1c9e4e52
47462025-04-30T06:43:57.379ZDEBGtest_utilization (SecStore): recording saga event event_type = Succeeded(Object {"instance_id": String("d7bd2deb-ee27-4661-9e9a-524d734cd6ec"), "new_id": String("05e54cff-eed5-453a-98a1-297d8872176d"), "saga_params": Object {"boundary_switches": Array [], "create_params": Object {"anti_affinity_groups": Array [], "auto_restart_policy": Null, "boot_disk": Null, "description": String("instance \\"utilization-test-instance\\""), "disks": Array [], "external_ips": Array [], "hostname": String("the-host"), "memory": Number(1073741824), "name": String("utilization-test-instance"), "ncpus": Number(4), "network_interfaces": Object {"type": String("default")}, "ssh_public_keys": Array [], "start": Bool(true), "user_data": String("I2Nsb3VkLWNvbmZpZwpzeXN0ZW1faW5mbzoKICBkZWZhdWx0X3VzZXI6CiAgICBuYW1lOiBveGlkZQ==")}, "project_id": String("473b39f9-8669-45b3-b3d7-793f7e10fc31"), "serialized_authn": Object {"kind": Object {"Authenticated": Array [Object {"actor": Object {"SiloUser": Object {"silo_id": String("001de000-5110-4000-8000-000000000000"), "silo_user_id": String("001de000-05e4-4000-8000-000000004007")}}}, Object {"mapped_fleet_roles": Object {}}]}}}, "which": Number(18)}) node_id = 109 saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c
47472025-04-30T06:43:57.384ZDEBGtest_utilization (SEC): saga log event new_state = N110 started sec_id = ac55d8d9-a298-4534-898b-beee1c9e4e52
47482025-04-30T06:43:57.385ZDEBGtest_utilization (SecStore): recording saga event event_type = Started node_id = 110 saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c
47492025-04-30T06:43:57.390ZDEBGtest_utilization (SEC): saga log event new_state = N110 succeeded sec_id = ac55d8d9-a298-4534-898b-beee1c9e4e52
47502025-04-30T06:43:57.390ZDEBGtest_utilization (SecStore): recording saga event event_type = Succeeded(Null) node_id = 110 saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c
47512025-04-30T06:43:57.395ZDEBGtest_utilization (SEC): saga log event new_state = N111 started sec_id = ac55d8d9-a298-4534-898b-beee1c9e4e52
47522025-04-30T06:43:57.396ZDEBGtest_utilization (SecStore): recording saga event event_type = Started node_id = 111 saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c
47532025-04-30T06:43:57.408ZDEBGtest_utilization (SEC): saga log event new_state = N111 succeeded sec_id = ac55d8d9-a298-4534-898b-beee1c9e4e52
47542025-04-30T06:43:57.408ZDEBGtest_utilization (SecStore): recording saga event event_type = Succeeded(Null) node_id = 111 saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c
47552025-04-30T06:43:57.412ZDEBGtest_utilization (SEC): saga log event new_state = N112 started sec_id = ac55d8d9-a298-4534-898b-beee1c9e4e52
47562025-04-30T06:43:57.412ZDEBGtest_utilization (SecStore): recording saga event event_type = Started node_id = 112 saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c
47572025-04-30T06:43:57.416ZDEBGtest_utilization (SEC): saga log event new_state = N112 succeeded sec_id = ac55d8d9-a298-4534-898b-beee1c9e4e52
47582025-04-30T06:43:57.416ZDEBGtest_utilization (SecStore): recording saga event event_type = Succeeded(Null) node_id = 112 saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c
47592025-04-30T06:43:57.426ZDEBGtest_utilization (SEC): saga log event new_state = N113 started sec_id = ac55d8d9-a298-4534-898b-beee1c9e4e52
47602025-04-30T06:43:57.426ZDEBGtest_utilization (SecStore): recording saga event event_type = Started node_id = 113 saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c
47612025-04-30T06:43:57.429ZDEBGtest_utilization (SEC): saga log event new_state = N113 succeeded sec_id = ac55d8d9-a298-4534-898b-beee1c9e4e52
47622025-04-30T06:43:57.429ZDEBGtest_utilization (SecStore): recording saga event event_type = Succeeded(Object {"instance_id": String("d7bd2deb-ee27-4661-9e9a-524d734cd6ec"), "new_id": String("fdb16759-a0d9-45ed-ac3f-ea7a5f13ab24"), "saga_params": Object {"boundary_switches": Array [], "create_params": Object {"anti_affinity_groups": Array [], "auto_restart_policy": Null, "boot_disk": Null, "description": String("instance \\"utilization-test-instance\\""), "disks": Array [], "external_ips": Array [], "hostname": String("the-host"), "memory": Number(1073741824), "name": String("utilization-test-instance"), "ncpus": Number(4), "network_interfaces": Object {"type": String("default")}, "ssh_public_keys": Array [], "start": Bool(true), "user_data": String("I2Nsb3VkLWNvbmZpZwpzeXN0ZW1faW5mbzoKICBkZWZhdWx0X3VzZXI6CiAgICBuYW1lOiBveGlkZQ==")}, "project_id": String("473b39f9-8669-45b3-b3d7-793f7e10fc31"), "serialized_authn": Object {"kind": Object {"Authenticated": Array [Object {"actor": Object {"SiloUser": Object {"silo_id": String("001de000-5110-4000-8000-000000000000"), "silo_user_id": String("001de000-05e4-4000-8000-000000004007")}}}, Object {"mapped_fleet_roles": Object {}}]}}}, "which": Number(19)}) node_id = 113 saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c
47632025-04-30T06:43:57.433ZDEBGtest_utilization (SEC): saga log event new_state = N114 started sec_id = ac55d8d9-a298-4534-898b-beee1c9e4e52
47642025-04-30T06:43:57.433ZDEBGtest_utilization (SecStore): recording saga event event_type = Started node_id = 114 saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c
47652025-04-30T06:43:57.441ZDEBGtest_utilization (SEC): saga log event new_state = N114 succeeded sec_id = ac55d8d9-a298-4534-898b-beee1c9e4e52
47662025-04-30T06:43:57.441ZDEBGtest_utilization (SecStore): recording saga event event_type = Succeeded(Null) node_id = 114 saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c
47672025-04-30T06:43:57.450ZDEBGtest_utilization (SEC): saga log event new_state = N115 started sec_id = ac55d8d9-a298-4534-898b-beee1c9e4e52
47682025-04-30T06:43:57.450ZDEBGtest_utilization (SecStore): recording saga event event_type = Started node_id = 115 saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c
47692025-04-30T06:43:57.455ZDEBGtest_utilization (SEC): saga log event new_state = N115 succeeded sec_id = ac55d8d9-a298-4534-898b-beee1c9e4e52
47702025-04-30T06:43:57.455ZDEBGtest_utilization (SecStore): recording saga event event_type = Succeeded(Null) node_id = 115 saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c
47712025-04-30T06:43:57.458ZWARNtest_utilization (results-sink-cluster): failed to insert some results into metric DB collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 error = Failed to check out connection to database: No backends found for this service file = oximeter/collector/src/results_sink.rs:92
47722025-04-30T06:43:57.460ZDEBGtest_utilization (SEC): saga log event new_state = N116 started sec_id = ac55d8d9-a298-4534-898b-beee1c9e4e52
47732025-04-30T06:43:57.460ZDEBGtest_utilization (SecStore): recording saga event event_type = Started node_id = 116 saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c
47742025-04-30T06:43:57.468ZDEBGtest_utilization (SEC): saga log event new_state = N116 succeeded sec_id = ac55d8d9-a298-4534-898b-beee1c9e4e52
47752025-04-30T06:43:57.468ZDEBGtest_utilization (SecStore): recording saga event event_type = Succeeded(Null) node_id = 116 saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c
47762025-04-30T06:43:57.472ZDEBGtest_utilization (SEC): saga log event new_state = N117 started sec_id = ac55d8d9-a298-4534-898b-beee1c9e4e52
47772025-04-30T06:43:57.472ZDEBGtest_utilization (SecStore): recording saga event event_type = Started node_id = 117 saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c
47782025-04-30T06:43:57.475ZDEBGtest_utilization (SEC): saga log event new_state = N117 succeeded sec_id = ac55d8d9-a298-4534-898b-beee1c9e4e52
47792025-04-30T06:43:57.475ZDEBGtest_utilization (SecStore): recording saga event event_type = Succeeded(Object {"instance_id": String("d7bd2deb-ee27-4661-9e9a-524d734cd6ec"), "new_id": String("b19a9736-163c-49c2-8115-ee28cc17761a"), "saga_params": Object {"boundary_switches": Array [], "create_params": Object {"anti_affinity_groups": Array [], "auto_restart_policy": Null, "boot_disk": Null, "description": String("instance \\"utilization-test-instance\\""), "disks": Array [], "external_ips": Array [], "hostname": String("the-host"), "memory": Number(1073741824), "name": String("utilization-test-instance"), "ncpus": Number(4), "network_interfaces": Object {"type": String("default")}, "ssh_public_keys": Array [], "start": Bool(true), "user_data": String("I2Nsb3VkLWNvbmZpZwpzeXN0ZW1faW5mbzoKICBkZWZhdWx0X3VzZXI6CiAgICBuYW1lOiBveGlkZQ==")}, "project_id": String("473b39f9-8669-45b3-b3d7-793f7e10fc31"), "serialized_authn": Object {"kind": Object {"Authenticated": Array [Object {"actor": Object {"SiloUser": Object {"silo_id": String("001de000-5110-4000-8000-000000000000"), "silo_user_id": String("001de000-05e4-4000-8000-000000004007")}}}, Object {"mapped_fleet_roles": Object {}}]}}}, "which": Number(20)}) node_id = 117 saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c
47802025-04-30T06:43:57.478ZDEBGtest_utilization (SEC): saga log event new_state = N118 started sec_id = ac55d8d9-a298-4534-898b-beee1c9e4e52
47812025-04-30T06:43:57.478ZDEBGtest_utilization (SecStore): recording saga event event_type = Started node_id = 118 saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c
47822025-04-30T06:43:57.480ZDEBGtest_utilization (SEC): saga log event new_state = N118 succeeded sec_id = ac55d8d9-a298-4534-898b-beee1c9e4e52
47832025-04-30T06:43:57.481ZDEBGtest_utilization (SecStore): recording saga event event_type = Succeeded(Null) node_id = 118 saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c
47842025-04-30T06:43:57.483ZDEBGtest_utilization (SEC): saga log event new_state = N119 started sec_id = ac55d8d9-a298-4534-898b-beee1c9e4e52
47852025-04-30T06:43:57.483ZDEBGtest_utilization (SecStore): recording saga event event_type = Started node_id = 119 saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c
47862025-04-30T06:43:57.487ZDEBGtest_utilization (SEC): saga log event new_state = N119 succeeded sec_id = ac55d8d9-a298-4534-898b-beee1c9e4e52
47872025-04-30T06:43:57.487ZDEBGtest_utilization (SecStore): recording saga event event_type = Succeeded(Null) node_id = 119 saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c
47882025-04-30T06:43:57.489ZDEBGtest_utilization (SEC): saga log event new_state = N120 started sec_id = ac55d8d9-a298-4534-898b-beee1c9e4e52
47892025-04-30T06:43:57.489ZDEBGtest_utilization (SecStore): recording saga event event_type = Started node_id = 120 saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c
47902025-04-30T06:43:57.499ZDEBGtest_utilization (SEC): saga log event new_state = N120 succeeded sec_id = ac55d8d9-a298-4534-898b-beee1c9e4e52
47912025-04-30T06:43:57.499ZDEBGtest_utilization (SecStore): recording saga event event_type = Succeeded(Null) node_id = 120 saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c
47922025-04-30T06:43:57.509ZDEBGtest_utilization (SEC): saga log event new_state = N121 started sec_id = ac55d8d9-a298-4534-898b-beee1c9e4e52
47932025-04-30T06:43:57.509ZDEBGtest_utilization (SecStore): recording saga event event_type = Started node_id = 121 saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c
47942025-04-30T06:43:57.512ZDEBGtest_utilization (SEC): saga log event new_state = N121 succeeded sec_id = ac55d8d9-a298-4534-898b-beee1c9e4e52
47952025-04-30T06:43:57.512ZDEBGtest_utilization (SecStore): recording saga event event_type = Succeeded(Object {"instance_id": String("d7bd2deb-ee27-4661-9e9a-524d734cd6ec"), "new_id": String("3e95bbf0-31cc-42db-9c13-91a80858663c"), "saga_params": Object {"boundary_switches": Array [], "create_params": Object {"anti_affinity_groups": Array [], "auto_restart_policy": Null, "boot_disk": Null, "description": String("instance \\"utilization-test-instance\\""), "disks": Array [], "external_ips": Array [], "hostname": String("the-host"), "memory": Number(1073741824), "name": String("utilization-test-instance"), "ncpus": Number(4), "network_interfaces": Object {"type": String("default")}, "ssh_public_keys": Array [], "start": Bool(true), "user_data": String("I2Nsb3VkLWNvbmZpZwpzeXN0ZW1faW5mbzoKICBkZWZhdWx0X3VzZXI6CiAgICBuYW1lOiBveGlkZQ==")}, "project_id": String("473b39f9-8669-45b3-b3d7-793f7e10fc31"), "serialized_authn": Object {"kind": Object {"Authenticated": Array [Object {"actor": Object {"SiloUser": Object {"silo_id": String("001de000-5110-4000-8000-000000000000"), "silo_user_id": String("001de000-05e4-4000-8000-000000004007")}}}, Object {"mapped_fleet_roles": Object {}}]}}}, "which": Number(21)}) node_id = 121 saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c
47962025-04-30T06:43:57.515ZDEBGtest_utilization (SEC): saga log event new_state = N122 started sec_id = ac55d8d9-a298-4534-898b-beee1c9e4e52
47972025-04-30T06:43:57.515ZDEBGtest_utilization (SecStore): recording saga event event_type = Started node_id = 122 saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c
47982025-04-30T06:43:57.518ZDEBGtest_utilization (SEC): saga log event new_state = N122 succeeded sec_id = ac55d8d9-a298-4534-898b-beee1c9e4e52
47992025-04-30T06:43:57.518ZDEBGtest_utilization (SecStore): recording saga event event_type = Succeeded(Null) node_id = 122 saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c
48002025-04-30T06:43:57.521ZDEBGtest_utilization (SEC): saga log event new_state = N123 started sec_id = ac55d8d9-a298-4534-898b-beee1c9e4e52
48012025-04-30T06:43:57.521ZDEBGtest_utilization (SecStore): recording saga event event_type = Started node_id = 123 saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c
48022025-04-30T06:43:57.528ZDEBGtest_utilization (SEC): saga log event new_state = N123 succeeded sec_id = ac55d8d9-a298-4534-898b-beee1c9e4e52
48032025-04-30T06:43:57.529ZDEBGtest_utilization (SecStore): recording saga event event_type = Succeeded(Null) node_id = 123 saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c
48042025-04-30T06:43:57.531ZDEBGtest_utilization (SEC): saga log event new_state = N124 started sec_id = ac55d8d9-a298-4534-898b-beee1c9e4e52
48052025-04-30T06:43:57.531ZDEBGtest_utilization (SecStore): recording saga event event_type = Started node_id = 124 saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c
48062025-04-30T06:43:57.535ZDEBGtest_utilization (SEC): saga log event new_state = N124 succeeded sec_id = ac55d8d9-a298-4534-898b-beee1c9e4e52
48072025-04-30T06:43:57.535ZDEBGtest_utilization (SecStore): recording saga event event_type = Succeeded(Null) node_id = 124 saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c
48082025-04-30T06:43:57.539ZDEBGtest_utilization (SEC): saga log event new_state = N125 started sec_id = ac55d8d9-a298-4534-898b-beee1c9e4e52
48092025-04-30T06:43:57.539ZDEBGtest_utilization (SecStore): recording saga event event_type = Started node_id = 125 saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c
48102025-04-30T06:43:57.543ZDEBGtest_utilization (SEC): saga log event new_state = N125 succeeded sec_id = ac55d8d9-a298-4534-898b-beee1c9e4e52
48112025-04-30T06:43:57.543ZDEBGtest_utilization (SecStore): recording saga event event_type = Succeeded(Object {"instance_id": String("d7bd2deb-ee27-4661-9e9a-524d734cd6ec"), "new_id": String("a4cb59af-13e0-43d4-9e6a-cb804af95f94"), "saga_params": Object {"boundary_switches": Array [], "create_params": Object {"anti_affinity_groups": Array [], "auto_restart_policy": Null, "boot_disk": Null, "description": String("instance \\"utilization-test-instance\\""), "disks": Array [], "external_ips": Array [], "hostname": String("the-host"), "memory": Number(1073741824), "name": String("utilization-test-instance"), "ncpus": Number(4), "network_interfaces": Object {"type": String("default")}, "ssh_public_keys": Array [], "start": Bool(true), "user_data": String("I2Nsb3VkLWNvbmZpZwpzeXN0ZW1faW5mbzoKICBkZWZhdWx0X3VzZXI6CiAgICBuYW1lOiBveGlkZQ==")}, "project_id": String("473b39f9-8669-45b3-b3d7-793f7e10fc31"), "serialized_authn": Object {"kind": Object {"Authenticated": Array [Object {"actor": Object {"SiloUser": Object {"silo_id": String("001de000-5110-4000-8000-000000000000"), "silo_user_id": String("001de000-05e4-4000-8000-000000004007")}}}, Object {"mapped_fleet_roles": Object {}}]}}}, "which": Number(22)}) node_id = 125 saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c
48122025-04-30T06:43:57.546ZDEBGtest_utilization (SEC): saga log event new_state = N126 started sec_id = ac55d8d9-a298-4534-898b-beee1c9e4e52
48132025-04-30T06:43:57.546ZDEBGtest_utilization (SecStore): recording saga event event_type = Started node_id = 126 saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c
48142025-04-30T06:43:57.550ZDEBGtest_utilization (SEC): saga log event new_state = N126 succeeded sec_id = ac55d8d9-a298-4534-898b-beee1c9e4e52
48152025-04-30T06:43:57.550ZDEBGtest_utilization (SecStore): recording saga event event_type = Succeeded(Null) node_id = 126 saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c
48162025-04-30T06:43:57.553ZDEBGtest_utilization (SEC): saga log event new_state = N127 started sec_id = ac55d8d9-a298-4534-898b-beee1c9e4e52
48172025-04-30T06:43:57.553ZDEBGtest_utilization (SecStore): recording saga event event_type = Started node_id = 127 saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c
48182025-04-30T06:43:57.556ZDEBGtest_utilization (SEC): saga log event new_state = N127 succeeded sec_id = ac55d8d9-a298-4534-898b-beee1c9e4e52
48192025-04-30T06:43:57.556ZDEBGtest_utilization (SecStore): recording saga event event_type = Succeeded(Null) node_id = 127 saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c
48202025-04-30T06:43:57.560ZDEBGtest_utilization (SEC): saga log event new_state = N128 started sec_id = ac55d8d9-a298-4534-898b-beee1c9e4e52
48212025-04-30T06:43:57.560ZDEBGtest_utilization (SecStore): recording saga event event_type = Started node_id = 128 saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c
48222025-04-30T06:43:57.563ZDEBGtest_utilization (SEC): saga log event new_state = N128 succeeded sec_id = ac55d8d9-a298-4534-898b-beee1c9e4e52
48232025-04-30T06:43:57.563ZDEBGtest_utilization (SecStore): recording saga event event_type = Succeeded(Null) node_id = 128 saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c
48242025-04-30T06:43:57.565ZDEBGtest_utilization (SEC): saga log event new_state = N129 started sec_id = ac55d8d9-a298-4534-898b-beee1c9e4e52
48252025-04-30T06:43:57.565ZDEBGtest_utilization (SecStore): recording saga event event_type = Started node_id = 129 saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c
48262025-04-30T06:43:57.572ZDEBGtest_utilization (SEC): saga log event new_state = N129 succeeded sec_id = ac55d8d9-a298-4534-898b-beee1c9e4e52
48272025-04-30T06:43:57.572ZDEBGtest_utilization (SecStore): recording saga event event_type = Succeeded(Object {"instance_id": String("d7bd2deb-ee27-4661-9e9a-524d734cd6ec"), "new_id": String("f4bf029b-6ff1-4269-a3aa-78663047295b"), "saga_params": Object {"boundary_switches": Array [], "create_params": Object {"anti_affinity_groups": Array [], "auto_restart_policy": Null, "boot_disk": Null, "description": String("instance \\"utilization-test-instance\\""), "disks": Array [], "external_ips": Array [], "hostname": String("the-host"), "memory": Number(1073741824), "name": String("utilization-test-instance"), "ncpus": Number(4), "network_interfaces": Object {"type": String("default")}, "ssh_public_keys": Array [], "start": Bool(true), "user_data": String("I2Nsb3VkLWNvbmZpZwpzeXN0ZW1faW5mbzoKICBkZWZhdWx0X3VzZXI6CiAgICBuYW1lOiBveGlkZQ==")}, "project_id": String("473b39f9-8669-45b3-b3d7-793f7e10fc31"), "serialized_authn": Object {"kind": Object {"Authenticated": Array [Object {"actor": Object {"SiloUser": Object {"silo_id": String("001de000-5110-4000-8000-000000000000"), "silo_user_id": String("001de000-05e4-4000-8000-000000004007")}}}, Object {"mapped_fleet_roles": Object {}}]}}}, "which": Number(23)}) node_id = 129 saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c
48282025-04-30T06:43:57.587ZDEBGtest_utilization (SEC): saga log event new_state = N130 started sec_id = ac55d8d9-a298-4534-898b-beee1c9e4e52
48292025-04-30T06:43:57.587ZDEBGtest_utilization (SecStore): recording saga event event_type = Started node_id = 130 saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c
48302025-04-30T06:43:57.600ZDEBGtest_utilization (SEC): saga log event new_state = N130 succeeded sec_id = ac55d8d9-a298-4534-898b-beee1c9e4e52
48312025-04-30T06:43:57.601ZDEBGtest_utilization (SecStore): recording saga event event_type = Succeeded(Null) node_id = 130 saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c
48322025-04-30T06:43:57.605ZDEBGtest_utilization (SEC): saga log event new_state = N131 started sec_id = ac55d8d9-a298-4534-898b-beee1c9e4e52
48332025-04-30T06:43:57.605ZDEBGtest_utilization (SecStore): recording saga event event_type = Started node_id = 131 saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c
48342025-04-30T06:43:57.610ZDEBGtest_utilization (SEC): saga log event new_state = N131 succeeded sec_id = ac55d8d9-a298-4534-898b-beee1c9e4e52
48352025-04-30T06:43:57.611ZDEBGtest_utilization (SecStore): recording saga event event_type = Succeeded(Null) node_id = 131 saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c
48362025-04-30T06:43:57.615ZDEBGtest_utilization (SEC): saga log event new_state = N132 started sec_id = ac55d8d9-a298-4534-898b-beee1c9e4e52
48372025-04-30T06:43:57.615ZDEBGtest_utilization (SecStore): recording saga event event_type = Started node_id = 132 saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c
48382025-04-30T06:43:57.618ZDEBGtest_utilization (SEC): saga log event new_state = N132 succeeded sec_id = ac55d8d9-a298-4534-898b-beee1c9e4e52
48392025-04-30T06:43:57.618ZDEBGtest_utilization (SecStore): recording saga event event_type = Succeeded(Null) node_id = 132 saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c
48402025-04-30T06:43:57.622ZDEBGtest_utilization (SEC): saga log event new_state = N133 started sec_id = ac55d8d9-a298-4534-898b-beee1c9e4e52
48412025-04-30T06:43:57.622ZDEBGtest_utilization (SecStore): recording saga event event_type = Started node_id = 133 saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c
48422025-04-30T06:43:57.625ZDEBGtest_utilization (SEC): saga log event new_state = N133 succeeded sec_id = ac55d8d9-a298-4534-898b-beee1c9e4e52
48432025-04-30T06:43:57.625ZDEBGtest_utilization (SecStore): recording saga event event_type = Succeeded(Object {"instance_id": String("d7bd2deb-ee27-4661-9e9a-524d734cd6ec"), "new_id": String("34abc739-27ab-4357-820a-4bd83d55c78b"), "saga_params": Object {"boundary_switches": Array [], "create_params": Object {"anti_affinity_groups": Array [], "auto_restart_policy": Null, "boot_disk": Null, "description": String("instance \\"utilization-test-instance\\""), "disks": Array [], "external_ips": Array [], "hostname": String("the-host"), "memory": Number(1073741824), "name": String("utilization-test-instance"), "ncpus": Number(4), "network_interfaces": Object {"type": String("default")}, "ssh_public_keys": Array [], "start": Bool(true), "user_data": String("I2Nsb3VkLWNvbmZpZwpzeXN0ZW1faW5mbzoKICBkZWZhdWx0X3VzZXI6CiAgICBuYW1lOiBveGlkZQ==")}, "project_id": String("473b39f9-8669-45b3-b3d7-793f7e10fc31"), "serialized_authn": Object {"kind": Object {"Authenticated": Array [Object {"actor": Object {"SiloUser": Object {"silo_id": String("001de000-5110-4000-8000-000000000000"), "silo_user_id": String("001de000-05e4-4000-8000-000000004007")}}}, Object {"mapped_fleet_roles": Object {}}]}}}, "which": Number(24)}) node_id = 133 saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c
48442025-04-30T06:43:57.629ZDEBGtest_utilization (SEC): saga log event new_state = N134 started sec_id = ac55d8d9-a298-4534-898b-beee1c9e4e52
48452025-04-30T06:43:57.629ZDEBGtest_utilization (SecStore): recording saga event event_type = Started node_id = 134 saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c
48462025-04-30T06:43:57.633ZDEBGtest_utilization (SEC): saga log event new_state = N134 succeeded sec_id = ac55d8d9-a298-4534-898b-beee1c9e4e52
48472025-04-30T06:43:57.633ZDEBGtest_utilization (SecStore): recording saga event event_type = Succeeded(Null) node_id = 134 saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c
48482025-04-30T06:43:57.637ZDEBGtest_utilization (SEC): saga log event new_state = N135 started sec_id = ac55d8d9-a298-4534-898b-beee1c9e4e52
48492025-04-30T06:43:57.637ZDEBGtest_utilization (SecStore): recording saga event event_type = Started node_id = 135 saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c
48502025-04-30T06:43:57.639ZDEBGtest_utilization (SEC): saga log event new_state = N135 succeeded sec_id = ac55d8d9-a298-4534-898b-beee1c9e4e52
48512025-04-30T06:43:57.639ZDEBGtest_utilization (SecStore): recording saga event event_type = Succeeded(Null) node_id = 135 saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c
48522025-04-30T06:43:57.643ZDEBGtest_utilization (SEC): saga log event new_state = N136 started sec_id = ac55d8d9-a298-4534-898b-beee1c9e4e52
48532025-04-30T06:43:57.643ZDEBGtest_utilization (SecStore): recording saga event event_type = Started node_id = 136 saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c
48542025-04-30T06:43:57.646ZDEBGtest_utilization (SEC): saga log event new_state = N136 succeeded sec_id = ac55d8d9-a298-4534-898b-beee1c9e4e52
48552025-04-30T06:43:57.646ZDEBGtest_utilization (SecStore): recording saga event event_type = Succeeded(Null) node_id = 136 saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c
48562025-04-30T06:43:57.649ZDEBGtest_utilization (SEC): saga log event new_state = N137 started sec_id = ac55d8d9-a298-4534-898b-beee1c9e4e52
48572025-04-30T06:43:57.649ZDEBGtest_utilization (SecStore): recording saga event event_type = Started node_id = 137 saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c
48582025-04-30T06:43:57.652ZDEBGtest_utilization (SEC): saga log event new_state = N137 succeeded sec_id = ac55d8d9-a298-4534-898b-beee1c9e4e52
48592025-04-30T06:43:57.652ZDEBGtest_utilization (SecStore): recording saga event event_type = Succeeded(Object {"instance_id": String("d7bd2deb-ee27-4661-9e9a-524d734cd6ec"), "new_id": String("c4c3995e-045b-4152-ad96-930a44eb1c91"), "saga_params": Object {"boundary_switches": Array [], "create_params": Object {"anti_affinity_groups": Array [], "auto_restart_policy": Null, "boot_disk": Null, "description": String("instance \\"utilization-test-instance\\""), "disks": Array [], "external_ips": Array [], "hostname": String("the-host"), "memory": Number(1073741824), "name": String("utilization-test-instance"), "ncpus": Number(4), "network_interfaces": Object {"type": String("default")}, "ssh_public_keys": Array [], "start": Bool(true), "user_data": String("I2Nsb3VkLWNvbmZpZwpzeXN0ZW1faW5mbzoKICBkZWZhdWx0X3VzZXI6CiAgICBuYW1lOiBveGlkZQ==")}, "project_id": String("473b39f9-8669-45b3-b3d7-793f7e10fc31"), "serialized_authn": Object {"kind": Object {"Authenticated": Array [Object {"actor": Object {"SiloUser": Object {"silo_id": String("001de000-5110-4000-8000-000000000000"), "silo_user_id": String("001de000-05e4-4000-8000-000000004007")}}}, Object {"mapped_fleet_roles": Object {}}]}}}, "which": Number(25)}) node_id = 137 saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c
48602025-04-30T06:43:57.655ZDEBGtest_utilization (SEC): saga log event new_state = N138 started sec_id = ac55d8d9-a298-4534-898b-beee1c9e4e52
48612025-04-30T06:43:57.655ZDEBGtest_utilization (SecStore): recording saga event event_type = Started node_id = 138 saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c
48622025-04-30T06:43:57.657ZDEBGtest_utilization (SEC): saga log event new_state = N138 succeeded sec_id = ac55d8d9-a298-4534-898b-beee1c9e4e52
48632025-04-30T06:43:57.657ZDEBGtest_utilization (SecStore): recording saga event event_type = Succeeded(Null) node_id = 138 saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c
48642025-04-30T06:43:57.661ZDEBGtest_utilization (SEC): saga log event new_state = N139 started sec_id = ac55d8d9-a298-4534-898b-beee1c9e4e52
48652025-04-30T06:43:57.661ZDEBGtest_utilization (SecStore): recording saga event event_type = Started node_id = 139 saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c
48662025-04-30T06:43:57.666ZDEBGtest_utilization (SEC): saga log event new_state = N139 succeeded sec_id = ac55d8d9-a298-4534-898b-beee1c9e4e52
48672025-04-30T06:43:57.666ZDEBGtest_utilization (SecStore): recording saga event event_type = Succeeded(Null) node_id = 139 saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c
48682025-04-30T06:43:57.669ZDEBGtest_utilization (SEC): saga log event new_state = N140 started sec_id = ac55d8d9-a298-4534-898b-beee1c9e4e52
48692025-04-30T06:43:57.669ZDEBGtest_utilization (SecStore): recording saga event event_type = Started node_id = 140 saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c
48702025-04-30T06:43:57.672ZDEBGtest_utilization (SEC): saga log event new_state = N140 succeeded sec_id = ac55d8d9-a298-4534-898b-beee1c9e4e52
48712025-04-30T06:43:57.672ZDEBGtest_utilization (SecStore): recording saga event event_type = Succeeded(Null) node_id = 140 saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c
48722025-04-30T06:43:57.677ZDEBGtest_utilization (SEC): saga log event new_state = N141 started sec_id = ac55d8d9-a298-4534-898b-beee1c9e4e52
48732025-04-30T06:43:57.677ZDEBGtest_utilization (SecStore): recording saga event event_type = Started node_id = 141 saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c
48742025-04-30T06:43:57.679ZDEBGtest_utilization (SEC): saga log event new_state = N141 succeeded sec_id = ac55d8d9-a298-4534-898b-beee1c9e4e52
48752025-04-30T06:43:57.680ZDEBGtest_utilization (SecStore): recording saga event event_type = Succeeded(Object {"instance_id": String("d7bd2deb-ee27-4661-9e9a-524d734cd6ec"), "new_id": String("aa70062c-595c-4b83-8d78-1efc9c2f5978"), "saga_params": Object {"boundary_switches": Array [], "create_params": Object {"anti_affinity_groups": Array [], "auto_restart_policy": Null, "boot_disk": Null, "description": String("instance \\"utilization-test-instance\\""), "disks": Array [], "external_ips": Array [], "hostname": String("the-host"), "memory": Number(1073741824), "name": String("utilization-test-instance"), "ncpus": Number(4), "network_interfaces": Object {"type": String("default")}, "ssh_public_keys": Array [], "start": Bool(true), "user_data": String("I2Nsb3VkLWNvbmZpZwpzeXN0ZW1faW5mbzoKICBkZWZhdWx0X3VzZXI6CiAgICBuYW1lOiBveGlkZQ==")}, "project_id": String("473b39f9-8669-45b3-b3d7-793f7e10fc31"), "serialized_authn": Object {"kind": Object {"Authenticated": Array [Object {"actor": Object {"SiloUser": Object {"silo_id": String("001de000-5110-4000-8000-000000000000"), "silo_user_id": String("001de000-05e4-4000-8000-000000004007")}}}, Object {"mapped_fleet_roles": Object {}}]}}}, "which": Number(26)}) node_id = 141 saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c
48762025-04-30T06:43:57.683ZDEBGtest_utilization (SEC): saga log event new_state = N142 started sec_id = ac55d8d9-a298-4534-898b-beee1c9e4e52
48772025-04-30T06:43:57.683ZDEBGtest_utilization (SecStore): recording saga event event_type = Started node_id = 142 saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c
48782025-04-30T06:43:57.686ZDEBGtest_utilization (SEC): saga log event new_state = N142 succeeded sec_id = ac55d8d9-a298-4534-898b-beee1c9e4e52
48792025-04-30T06:43:57.686ZDEBGtest_utilization (SecStore): recording saga event event_type = Succeeded(Null) node_id = 142 saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c
48802025-04-30T06:43:57.690ZDEBGtest_utilization (SEC): saga log event new_state = N143 started sec_id = ac55d8d9-a298-4534-898b-beee1c9e4e52
48812025-04-30T06:43:57.690ZDEBGtest_utilization (SecStore): recording saga event event_type = Started node_id = 143 saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c
48822025-04-30T06:43:57.694ZDEBGtest_utilization (SEC): saga log event new_state = N143 succeeded sec_id = ac55d8d9-a298-4534-898b-beee1c9e4e52
48832025-04-30T06:43:57.694ZDEBGtest_utilization (SecStore): recording saga event event_type = Succeeded(Null) node_id = 143 saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c
48842025-04-30T06:43:57.698ZDEBGtest_utilization (SEC): saga log event new_state = N144 started sec_id = ac55d8d9-a298-4534-898b-beee1c9e4e52
48852025-04-30T06:43:57.698ZDEBGtest_utilization (SecStore): recording saga event event_type = Started node_id = 144 saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c
48862025-04-30T06:43:57.701ZDEBGtest_utilization (SEC): saga log event new_state = N144 succeeded sec_id = ac55d8d9-a298-4534-898b-beee1c9e4e52
48872025-04-30T06:43:57.701ZDEBGtest_utilization (SecStore): recording saga event event_type = Succeeded(Null) node_id = 144 saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c
48882025-04-30T06:43:57.704ZDEBGtest_utilization (SEC): saga log event new_state = N145 started sec_id = ac55d8d9-a298-4534-898b-beee1c9e4e52
48892025-04-30T06:43:57.704ZDEBGtest_utilization (SecStore): recording saga event event_type = Started node_id = 145 saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c
48902025-04-30T06:43:57.707ZDEBGtest_utilization (SEC): saga log event new_state = N145 succeeded sec_id = ac55d8d9-a298-4534-898b-beee1c9e4e52
48912025-04-30T06:43:57.707ZDEBGtest_utilization (SecStore): recording saga event event_type = Succeeded(Object {"instance_id": String("d7bd2deb-ee27-4661-9e9a-524d734cd6ec"), "new_id": String("25bbbf93-a75b-4a07-b175-ad5ff6de72e6"), "saga_params": Object {"boundary_switches": Array [], "create_params": Object {"anti_affinity_groups": Array [], "auto_restart_policy": Null, "boot_disk": Null, "description": String("instance \\"utilization-test-instance\\""), "disks": Array [], "external_ips": Array [], "hostname": String("the-host"), "memory": Number(1073741824), "name": String("utilization-test-instance"), "ncpus": Number(4), "network_interfaces": Object {"type": String("default")}, "ssh_public_keys": Array [], "start": Bool(true), "user_data": String("I2Nsb3VkLWNvbmZpZwpzeXN0ZW1faW5mbzoKICBkZWZhdWx0X3VzZXI6CiAgICBuYW1lOiBveGlkZQ==")}, "project_id": String("473b39f9-8669-45b3-b3d7-793f7e10fc31"), "serialized_authn": Object {"kind": Object {"Authenticated": Array [Object {"actor": Object {"SiloUser": Object {"silo_id": String("001de000-5110-4000-8000-000000000000"), "silo_user_id": String("001de000-05e4-4000-8000-000000004007")}}}, Object {"mapped_fleet_roles": Object {}}]}}}, "which": Number(27)}) node_id = 145 saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c
48922025-04-30T06:43:57.711ZDEBGtest_utilization (SEC): saga log event new_state = N146 started sec_id = ac55d8d9-a298-4534-898b-beee1c9e4e52
48932025-04-30T06:43:57.711ZDEBGtest_utilization (SecStore): recording saga event event_type = Started node_id = 146 saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c
48942025-04-30T06:43:57.715ZDEBGtest_utilization (SEC): saga log event new_state = N146 succeeded sec_id = ac55d8d9-a298-4534-898b-beee1c9e4e52
48952025-04-30T06:43:57.715ZDEBGtest_utilization (SecStore): recording saga event event_type = Succeeded(Null) node_id = 146 saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c
48962025-04-30T06:43:57.718ZDEBGtest_utilization (SEC): saga log event new_state = N147 started sec_id = ac55d8d9-a298-4534-898b-beee1c9e4e52
48972025-04-30T06:43:57.719ZDEBGtest_utilization (SecStore): recording saga event event_type = Started node_id = 147 saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c
48982025-04-30T06:43:57.722ZDEBGtest_utilization (SEC): saga log event new_state = N147 succeeded sec_id = ac55d8d9-a298-4534-898b-beee1c9e4e52
48992025-04-30T06:43:57.722ZDEBGtest_utilization (SecStore): recording saga event event_type = Succeeded(Null) node_id = 147 saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c
49002025-04-30T06:43:57.725ZDEBGtest_utilization (SEC): saga log event new_state = N148 started sec_id = ac55d8d9-a298-4534-898b-beee1c9e4e52
49012025-04-30T06:43:57.725ZDEBGtest_utilization (SecStore): recording saga event event_type = Started node_id = 148 saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c
49022025-04-30T06:43:57.728ZDEBGtest_utilization (SEC): saga log event new_state = N148 succeeded sec_id = ac55d8d9-a298-4534-898b-beee1c9e4e52
49032025-04-30T06:43:57.728ZDEBGtest_utilization (SecStore): recording saga event event_type = Succeeded(Null) node_id = 148 saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c
49042025-04-30T06:43:57.731ZDEBGtest_utilization (SEC): saga log event new_state = N149 started sec_id = ac55d8d9-a298-4534-898b-beee1c9e4e52
49052025-04-30T06:43:57.732ZDEBGtest_utilization (SecStore): recording saga event event_type = Started node_id = 149 saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c
49062025-04-30T06:43:57.734ZDEBGtest_utilization (SEC): saga log event new_state = N149 succeeded sec_id = ac55d8d9-a298-4534-898b-beee1c9e4e52
49072025-04-30T06:43:57.734ZDEBGtest_utilization (SecStore): recording saga event event_type = Succeeded(Object {"instance_id": String("d7bd2deb-ee27-4661-9e9a-524d734cd6ec"), "new_id": String("6047d897-d5be-45b8-93ff-0710079dfeae"), "saga_params": Object {"boundary_switches": Array [], "create_params": Object {"anti_affinity_groups": Array [], "auto_restart_policy": Null, "boot_disk": Null, "description": String("instance \\"utilization-test-instance\\""), "disks": Array [], "external_ips": Array [], "hostname": String("the-host"), "memory": Number(1073741824), "name": String("utilization-test-instance"), "ncpus": Number(4), "network_interfaces": Object {"type": String("default")}, "ssh_public_keys": Array [], "start": Bool(true), "user_data": String("I2Nsb3VkLWNvbmZpZwpzeXN0ZW1faW5mbzoKICBkZWZhdWx0X3VzZXI6CiAgICBuYW1lOiBveGlkZQ==")}, "project_id": String("473b39f9-8669-45b3-b3d7-793f7e10fc31"), "serialized_authn": Object {"kind": Object {"Authenticated": Array [Object {"actor": Object {"SiloUser": Object {"silo_id": String("001de000-5110-4000-8000-000000000000"), "silo_user_id": String("001de000-05e4-4000-8000-000000004007")}}}, Object {"mapped_fleet_roles": Object {}}]}}}, "which": Number(28)}) node_id = 149 saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c
49082025-04-30T06:43:57.737ZDEBGtest_utilization (SEC): saga log event new_state = N150 started sec_id = ac55d8d9-a298-4534-898b-beee1c9e4e52
49092025-04-30T06:43:57.737ZDEBGtest_utilization (SecStore): recording saga event event_type = Started node_id = 150 saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c
49102025-04-30T06:43:57.742ZDEBGtest_utilization (SEC): saga log event new_state = N150 succeeded sec_id = ac55d8d9-a298-4534-898b-beee1c9e4e52
49112025-04-30T06:43:57.742ZDEBGtest_utilization (SecStore): recording saga event event_type = Succeeded(Null) node_id = 150 saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c
49122025-04-30T06:43:57.746ZDEBGtest_utilization (SEC): saga log event new_state = N151 started sec_id = ac55d8d9-a298-4534-898b-beee1c9e4e52
49132025-04-30T06:43:57.746ZDEBGtest_utilization (SecStore): recording saga event event_type = Started node_id = 151 saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c
49142025-04-30T06:43:57.749ZDEBGtest_utilization (SEC): saga log event new_state = N151 succeeded sec_id = ac55d8d9-a298-4534-898b-beee1c9e4e52
49152025-04-30T06:43:57.749ZDEBGtest_utilization (SecStore): recording saga event event_type = Succeeded(Null) node_id = 151 saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c
49162025-04-30T06:43:57.755ZDEBGtest_utilization (SEC): saga log event new_state = N152 started sec_id = ac55d8d9-a298-4534-898b-beee1c9e4e52
49172025-04-30T06:43:57.755ZDEBGtest_utilization (SecStore): recording saga event event_type = Started node_id = 152 saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c
49182025-04-30T06:43:57.760ZDEBGtest_utilization (SEC): saga log event new_state = N152 succeeded sec_id = ac55d8d9-a298-4534-898b-beee1c9e4e52
49192025-04-30T06:43:57.760ZDEBGtest_utilization (SecStore): recording saga event event_type = Succeeded(Null) node_id = 152 saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c
49202025-04-30T06:43:57.764ZDEBGtest_utilization (SEC): saga log event new_state = N153 started sec_id = ac55d8d9-a298-4534-898b-beee1c9e4e52
49212025-04-30T06:43:57.764ZDEBGtest_utilization (SecStore): recording saga event event_type = Started node_id = 153 saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c
49222025-04-30T06:43:57.767ZDEBGtest_utilization (SEC): saga log event new_state = N153 succeeded sec_id = ac55d8d9-a298-4534-898b-beee1c9e4e52
49232025-04-30T06:43:57.767ZDEBGtest_utilization (SecStore): recording saga event event_type = Succeeded(Object {"instance_id": String("d7bd2deb-ee27-4661-9e9a-524d734cd6ec"), "new_id": String("fe58941b-b766-487e-812e-f5e79533a730"), "saga_params": Object {"boundary_switches": Array [], "create_params": Object {"anti_affinity_groups": Array [], "auto_restart_policy": Null, "boot_disk": Null, "description": String("instance \\"utilization-test-instance\\""), "disks": Array [], "external_ips": Array [], "hostname": String("the-host"), "memory": Number(1073741824), "name": String("utilization-test-instance"), "ncpus": Number(4), "network_interfaces": Object {"type": String("default")}, "ssh_public_keys": Array [], "start": Bool(true), "user_data": String("I2Nsb3VkLWNvbmZpZwpzeXN0ZW1faW5mbzoKICBkZWZhdWx0X3VzZXI6CiAgICBuYW1lOiBveGlkZQ==")}, "project_id": String("473b39f9-8669-45b3-b3d7-793f7e10fc31"), "serialized_authn": Object {"kind": Object {"Authenticated": Array [Object {"actor": Object {"SiloUser": Object {"silo_id": String("001de000-5110-4000-8000-000000000000"), "silo_user_id": String("001de000-05e4-4000-8000-000000004007")}}}, Object {"mapped_fleet_roles": Object {}}]}}}, "which": Number(29)}) node_id = 153 saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c
49242025-04-30T06:43:57.782ZDEBGtest_utilization (SEC): saga log event new_state = N154 started sec_id = ac55d8d9-a298-4534-898b-beee1c9e4e52
49252025-04-30T06:43:57.782ZDEBGtest_utilization (SecStore): recording saga event event_type = Started node_id = 154 saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c
49262025-04-30T06:43:57.788ZDEBGtest_utilization (SEC): saga log event new_state = N154 succeeded sec_id = ac55d8d9-a298-4534-898b-beee1c9e4e52
49272025-04-30T06:43:57.788ZDEBGtest_utilization (SecStore): recording saga event event_type = Succeeded(Null) node_id = 154 saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c
49282025-04-30T06:43:57.809ZDEBGtest_utilization (SEC): saga log event new_state = N155 started sec_id = ac55d8d9-a298-4534-898b-beee1c9e4e52
49292025-04-30T06:43:57.809ZDEBGtest_utilization (SecStore): recording saga event event_type = Started node_id = 155 saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c
49302025-04-30T06:43:57.832ZDEBGtest_utilization (SEC): saga log event new_state = N155 succeeded sec_id = ac55d8d9-a298-4534-898b-beee1c9e4e52
49312025-04-30T06:43:57.832ZDEBGtest_utilization (SecStore): recording saga event event_type = Succeeded(Null) node_id = 155 saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c
49322025-04-30T06:43:57.837ZDEBGtest_utilization (SEC): saga log event new_state = N156 started sec_id = ac55d8d9-a298-4534-898b-beee1c9e4e52
49332025-04-30T06:43:57.837ZDEBGtest_utilization (SecStore): recording saga event event_type = Started node_id = 156 saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c
49342025-04-30T06:43:57.843ZDEBGtest_utilization (SEC): saga log event new_state = N156 succeeded sec_id = ac55d8d9-a298-4534-898b-beee1c9e4e52
49352025-04-30T06:43:57.846ZDEBGtest_utilization (SecStore): recording saga event event_type = Succeeded(Null) node_id = 156 saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c
49362025-04-30T06:43:57.854ZDEBGtest_utilization (SEC): saga log event new_state = N157 started sec_id = ac55d8d9-a298-4534-898b-beee1c9e4e52
49372025-04-30T06:43:57.854ZDEBGtest_utilization (SecStore): recording saga event event_type = Started node_id = 157 saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c
49382025-04-30T06:43:57.858ZDEBGtest_utilization (SEC): saga log event new_state = N157 succeeded sec_id = ac55d8d9-a298-4534-898b-beee1c9e4e52
49392025-04-30T06:43:57.858ZDEBGtest_utilization (SecStore): recording saga event event_type = Succeeded(Object {"instance_id": String("d7bd2deb-ee27-4661-9e9a-524d734cd6ec"), "new_id": String("d4aeb710-e637-4ddd-b35b-dd02d5495d12"), "saga_params": Object {"boundary_switches": Array [], "create_params": Object {"anti_affinity_groups": Array [], "auto_restart_policy": Null, "boot_disk": Null, "description": String("instance \\"utilization-test-instance\\""), "disks": Array [], "external_ips": Array [], "hostname": String("the-host"), "memory": Number(1073741824), "name": String("utilization-test-instance"), "ncpus": Number(4), "network_interfaces": Object {"type": String("default")}, "ssh_public_keys": Array [], "start": Bool(true), "user_data": String("I2Nsb3VkLWNvbmZpZwpzeXN0ZW1faW5mbzoKICBkZWZhdWx0X3VzZXI6CiAgICBuYW1lOiBveGlkZQ==")}, "project_id": String("473b39f9-8669-45b3-b3d7-793f7e10fc31"), "serialized_authn": Object {"kind": Object {"Authenticated": Array [Object {"actor": Object {"SiloUser": Object {"silo_id": String("001de000-5110-4000-8000-000000000000"), "silo_user_id": String("001de000-05e4-4000-8000-000000004007")}}}, Object {"mapped_fleet_roles": Object {}}]}}}, "which": Number(30)}) node_id = 157 saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c
49402025-04-30T06:43:57.864ZDEBGtest_utilization (SEC): saga log event new_state = N158 started sec_id = ac55d8d9-a298-4534-898b-beee1c9e4e52
49412025-04-30T06:43:57.864ZDEBGtest_utilization (SecStore): recording saga event event_type = Started node_id = 158 saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c
49422025-04-30T06:43:57.870ZDEBGtest_utilization (SEC): saga log event new_state = N158 succeeded sec_id = ac55d8d9-a298-4534-898b-beee1c9e4e52
49432025-04-30T06:43:57.871ZDEBGtest_utilization (SecStore): recording saga event event_type = Succeeded(Null) node_id = 158 saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c
49442025-04-30T06:43:57.880ZDEBGtest_utilization (SEC): saga log event new_state = N159 started sec_id = ac55d8d9-a298-4534-898b-beee1c9e4e52
49452025-04-30T06:43:57.880ZDEBGtest_utilization (SecStore): recording saga event event_type = Started node_id = 159 saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c
49462025-04-30T06:43:57.897ZDEBGtest_utilization (SEC): saga log event new_state = N159 succeeded sec_id = ac55d8d9-a298-4534-898b-beee1c9e4e52
49472025-04-30T06:43:57.897ZDEBGtest_utilization (SecStore): recording saga event event_type = Succeeded(Null) node_id = 159 saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c
49482025-04-30T06:43:57.905ZDEBGtest_utilization (SEC): saga log event new_state = N160 started sec_id = ac55d8d9-a298-4534-898b-beee1c9e4e52
49492025-04-30T06:43:57.906ZDEBGtest_utilization (SecStore): recording saga event event_type = Started node_id = 160 saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c
49502025-04-30T06:43:57.914ZDEBGtest_utilization (SEC): saga log event new_state = N160 succeeded sec_id = ac55d8d9-a298-4534-898b-beee1c9e4e52
49512025-04-30T06:43:57.914ZDEBGtest_utilization (SecStore): recording saga event event_type = Succeeded(Null) node_id = 160 saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c
49522025-04-30T06:43:57.919ZDEBGtest_utilization (SEC): saga log event new_state = N161 started sec_id = ac55d8d9-a298-4534-898b-beee1c9e4e52
49532025-04-30T06:43:57.919ZDEBGtest_utilization (SecStore): recording saga event event_type = Started node_id = 161 saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c
49542025-04-30T06:43:57.930ZDEBGtest_utilization (SEC): saga log event new_state = N161 succeeded sec_id = ac55d8d9-a298-4534-898b-beee1c9e4e52
49552025-04-30T06:43:57.930ZDEBGtest_utilization (SecStore): recording saga event event_type = Succeeded(Object {"instance_id": String("d7bd2deb-ee27-4661-9e9a-524d734cd6ec"), "new_id": String("28acecea-be1f-451f-ac0d-066b7436b4d2"), "saga_params": Object {"boundary_switches": Array [], "create_params": Object {"anti_affinity_groups": Array [], "auto_restart_policy": Null, "boot_disk": Null, "description": String("instance \\"utilization-test-instance\\""), "disks": Array [], "external_ips": Array [], "hostname": String("the-host"), "memory": Number(1073741824), "name": String("utilization-test-instance"), "ncpus": Number(4), "network_interfaces": Object {"type": String("default")}, "ssh_public_keys": Array [], "start": Bool(true), "user_data": String("I2Nsb3VkLWNvbmZpZwpzeXN0ZW1faW5mbzoKICBkZWZhdWx0X3VzZXI6CiAgICBuYW1lOiBveGlkZQ==")}, "project_id": String("473b39f9-8669-45b3-b3d7-793f7e10fc31"), "serialized_authn": Object {"kind": Object {"Authenticated": Array [Object {"actor": Object {"SiloUser": Object {"silo_id": String("001de000-5110-4000-8000-000000000000"), "silo_user_id": String("001de000-05e4-4000-8000-000000004007")}}}, Object {"mapped_fleet_roles": Object {}}]}}}, "which": Number(31)}) node_id = 161 saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c
49562025-04-30T06:43:57.934ZDEBGtest_utilization (SEC): saga log event new_state = N162 started sec_id = ac55d8d9-a298-4534-898b-beee1c9e4e52
49572025-04-30T06:43:57.935ZDEBGtest_utilization (SecStore): recording saga event event_type = Started node_id = 162 saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c
49582025-04-30T06:43:57.947ZDEBGtest_utilization (SEC): saga log event new_state = N162 succeeded sec_id = ac55d8d9-a298-4534-898b-beee1c9e4e52
49592025-04-30T06:43:57.947ZDEBGtest_utilization (SecStore): recording saga event event_type = Succeeded(Null) node_id = 162 saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c
49602025-04-30T06:43:57.951ZDEBGtest_utilization (SEC): saga log event new_state = N163 started sec_id = ac55d8d9-a298-4534-898b-beee1c9e4e52
49612025-04-30T06:43:57.951ZDEBGtest_utilization (SecStore): recording saga event event_type = Started node_id = 163 saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c
49622025-04-30T06:43:57.954ZDEBGtest_utilization (SEC): saga log event new_state = N163 succeeded sec_id = ac55d8d9-a298-4534-898b-beee1c9e4e52
49632025-04-30T06:43:57.954ZDEBGtest_utilization (SecStore): recording saga event event_type = Succeeded(Null) node_id = 163 saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c
49642025-04-30T06:43:57.957ZINFOtest_utilization (oximeter-agent): refreshing list of producers from Nexus collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 file = oximeter/collector/src/agent.rs:562
49652025-04-30T06:43:57.957ZDEBGtest_utilization (oximeter): client request body = None method = GET uri = http://127.0.0.1:32956/metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?sort_by=id_ascending
49662025-04-30T06:43:57.958ZDEBGtest_utilization (SEC): saga log event new_state = N164 started sec_id = ac55d8d9-a298-4534-898b-beee1c9e4e52
49672025-04-30T06:43:57.958ZDEBGtest_utilization (SecStore): recording saga event event_type = Started node_id = 164 saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c
49682025-04-30T06:43:57.958ZTRACtest_utilization (dropshot_internal): incoming request local_addr = 127.0.0.1:32956 method = GET remote_addr = 127.0.0.1:55797 req_id = 1352a2e5-f950-48de-bded-3a29aa9b76cc uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?sort_by=id_ascending
49692025-04-30T06:43:57.959ZTRACtest_utilization (dropshot_internal): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = 127.0.0.1:32956 method = GET remote_addr = 127.0.0.1:55797 req_id = 1352a2e5-f950-48de-bded-3a29aa9b76cc resource = Database uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?sort_by=id_ascending
49702025-04-30T06:43:57.959ZDEBGtest_utilization (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = 127.0.0.1:32956 method = GET remote_addr = 127.0.0.1:55797 req_id = 1352a2e5-f950-48de-bded-3a29aa9b76cc roles = RoleSet { roles: {} } uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?sort_by=id_ascending
49712025-04-30T06:43:57.959ZDEBGtest_utilization (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = 127.0.0.1:32956 method = GET remote_addr = 127.0.0.1:55797 req_id = 1352a2e5-f950-48de-bded-3a29aa9b76cc resource = Database result = Ok(()) uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?sort_by=id_ascending
49722025-04-30T06:43:57.962ZDEBGtest_utilization (SEC): saga log event new_state = N164 succeeded sec_id = ac55d8d9-a298-4534-898b-beee1c9e4e52
49732025-04-30T06:43:57.962ZDEBGtest_utilization (SecStore): recording saga event event_type = Succeeded(Null) node_id = 164 saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c
49742025-04-30T06:43:57.965ZINFOtest_utilization (dropshot_internal): request completed latency_us = 6400 local_addr = 127.0.0.1:32956 method = GET remote_addr = 127.0.0.1:55797 req_id = 1352a2e5-f950-48de-bded-3a29aa9b76cc response_code = 200 uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?sort_by=id_ascending
49752025-04-30T06:43:57.970ZDEBGtest_utilization (SEC): saga log event new_state = N165 started sec_id = ac55d8d9-a298-4534-898b-beee1c9e4e52
49762025-04-30T06:43:57.970ZDEBGtest_utilization (SecStore): recording saga event event_type = Started node_id = 165 saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c
49772025-04-30T06:43:57.970ZDEBGtest_utilization (oximeter): client response result = Ok(Response { url: "http://127.0.0.1:32956/metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?sort_by=id_ascending", status: 200, headers: {"content-type": "application/json", "x-request-id": "1352a2e5-f950-48de-bded-3a29aa9b76cc", "content-length": "405", "date": "Wed, 30 Apr 2025 06:43:57 GMT"} })
49782025-04-30T06:43:57.971ZDEBGtest_utilization (oximeter): client request body = None method = GET uri = http://127.0.0.1:32956/metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJhYzU1ZDhkOS1hMjk4LTQ1MzQtODk4Yi1iZWVlMWM5ZTRlNTIifX0%3D
49792025-04-30T06:43:57.972ZTRACtest_utilization (dropshot_internal): incoming request local_addr = 127.0.0.1:32956 method = GET remote_addr = 127.0.0.1:55797 req_id = 2e128a75-4df0-47f8-99a1-d344590454da uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJhYzU1ZDhkOS1hMjk4LTQ1MzQtODk4Yi1iZWVlMWM5ZTRlNTIifX0%3D
49802025-04-30T06:43:57.972ZTRACtest_utilization (dropshot_internal): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = 127.0.0.1:32956 method = GET remote_addr = 127.0.0.1:55797 req_id = 2e128a75-4df0-47f8-99a1-d344590454da resource = Database uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJhYzU1ZDhkOS1hMjk4LTQ1MzQtODk4Yi1iZWVlMWM5ZTRlNTIifX0%3D
49812025-04-30T06:43:57.972ZDEBGtest_utilization (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = 127.0.0.1:32956 method = GET remote_addr = 127.0.0.1:55797 req_id = 2e128a75-4df0-47f8-99a1-d344590454da roles = RoleSet { roles: {} } uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJhYzU1ZDhkOS1hMjk4LTQ1MzQtODk4Yi1iZWVlMWM5ZTRlNTIifX0%3D
49822025-04-30T06:43:57.973ZDEBGtest_utilization (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = 127.0.0.1:32956 method = GET remote_addr = 127.0.0.1:55797 req_id = 2e128a75-4df0-47f8-99a1-d344590454da resource = Database result = Ok(()) uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJhYzU1ZDhkOS1hMjk4LTQ1MzQtODk4Yi1iZWVlMWM5ZTRlNTIifX0%3D
49832025-04-30T06:43:57.975ZDEBGtest_utilization (SEC): saga log event new_state = N165 succeeded sec_id = ac55d8d9-a298-4534-898b-beee1c9e4e52
49842025-04-30T06:43:57.975ZDEBGtest_utilization (SecStore): recording saga event event_type = Succeeded(Null) node_id = 165 saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c
49852025-04-30T06:43:57.978ZINFOtest_utilization (dropshot_internal): request completed latency_us = 6884 local_addr = 127.0.0.1:32956 method = GET remote_addr = 127.0.0.1:55797 req_id = 2e128a75-4df0-47f8-99a1-d344590454da response_code = 200 uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJhYzU1ZDhkOS1hMjk4LTQ1MzQtODk4Yi1iZWVlMWM5ZTRlNTIifX0%3D
49862025-04-30T06:43:57.979ZDEBGtest_utilization (SEC): saga log event new_state = N166 started sec_id = ac55d8d9-a298-4534-898b-beee1c9e4e52
49872025-04-30T06:43:57.979ZDEBGtest_utilization (SecStore): recording saga event event_type = Started node_id = 166 saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c
49882025-04-30T06:43:57.980ZDEBGtest_utilization (oximeter): client response result = Ok(Response { url: "http://127.0.0.1:32956/metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJhYzU1ZDhkOS1hMjk4LTQ1MzQtODk4Yi1iZWVlMWM5ZTRlNTIifX0%3D", status: 200, headers: {"content-type": "application/json", "x-request-id": "2e128a75-4df0-47f8-99a1-d344590454da", "content-length": "29", "date": "Wed, 30 Apr 2025 06:43:57 GMT"} })
49892025-04-30T06:43:57.980ZTRACtest_utilization (oximeter-agent): ignoring request to update existing metric producer, since the endpoint information is the same as the existing collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
49902025-04-30T06:43:57.980ZTRACtest_utilization (oximeter-agent): ignoring request to update existing metric producer, since the endpoint information is the same as the existing collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = ac55d8d9-a298-4534-898b-beee1c9e4e52
49912025-04-30T06:43:57.980ZINFOtest_utilization (oximeter-agent): refreshed list of producers from Nexus collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 file = oximeter/collector/src/agent.rs:637 n_current_tasks = 2 n_pruned_tasks = 0
49922025-04-30T06:43:58.009ZDEBGtest_utilization (collection-task): sent timer-based collection request to the collection task collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
49932025-04-30T06:43:58.009ZDEBGtest_utilization (collection-task): collecting from producer collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
49942025-04-30T06:43:58.009ZDEBGtest_utilization (SEC): saga log event new_state = N166 succeeded sec_id = ac55d8d9-a298-4534-898b-beee1c9e4e52
49952025-04-30T06:43:58.009ZDEBGtest_utilization (SecStore): recording saga event event_type = Succeeded(Null) node_id = 166 saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c
49962025-04-30T06:43:58.012ZTRACtest_utilization (collection-task): sent collection request to producer collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
49972025-04-30T06:43:58.013ZDEBGtest_utilization (collection-task): collected results from producer collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 n_results = 1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
49982025-04-30T06:43:58.013ZTRACtest_utilization (collection-task): forwarded results to main collection loop collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
49992025-04-30T06:43:58.013ZTRACtest_utilization (collection-task): top of inner collection loop, waiting for next request collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
50002025-04-30T06:43:58.014ZDEBGtest_utilization (SEC): saga log event new_state = N168 started sec_id = ac55d8d9-a298-4534-898b-beee1c9e4e52
50012025-04-30T06:43:58.014ZDEBGtest_utilization (SecStore): recording saga event event_type = Started node_id = 168 saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c
50022025-04-30T06:43:58.019ZDEBGtest_utilization (SEC): saga log event new_state = N168 succeeded sec_id = ac55d8d9-a298-4534-898b-beee1c9e4e52
50032025-04-30T06:43:58.019ZDEBGtest_utilization (SecStore): recording saga event event_type = Succeeded(Null) node_id = 168 saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c
50042025-04-30T06:43:58.023ZINFOtest_utilization (SEC): update for saga cached state new_state = Done saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c sec_id = ac55d8d9-a298-4534-898b-beee1c9e4e52
50052025-04-30T06:43:58.023ZINFOtest_utilization (SecStore): updating state new_state = done saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c
50062025-04-30T06:43:58.072ZINFOtest_utilization (SEC): saga finished result = success saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c saga_name = instance-create sec_id = ac55d8d9-a298-4534-898b-beee1c9e4e52
50072025-04-30T06:43:58.074ZINFOtest_utilization (SagaExecutor): saga finished saga_id = fbb67be9-6842-405a-8b7c-1b059fe7a14c saga_name = instance-create saga_result = SagaResult { saga_id: fbb67be9-6842-405a-8b7c-1b059fe7a14c, saga_log: SagaLog { saga_id: fbb67be9-6842-405a-8b7c-1b059fe7a14c, unwinding: false, events: [N167 started, N167 succeeded, N000 started, N000 succeeded, N001 started, N001 succeeded, N002 started, N002 succeeded, N003 started, N003 succeeded, N004 started, N004 succeeded, N005 started, N005 succeeded, N006 started, N006 succeeded, N007 started, N007 succeeded, N008 started, N008 succeeded, N009 started, N009 succeeded, N010 started, N010 succeeded, N011 started, N011 succeeded, N012 started, N012 succeeded, N013 started, N013 succeeded, N014 started, N014 succeeded, N015 started, N015 succeeded, N016 started, N016 succeeded, N017 started, N017 succeeded, N018 started, N018 succeeded, N019 started, N019 succeeded, N020 started, N020 succeeded, N021 started, N021 succeeded, N022 started, N022 succeeded, N023 started, N023 succeeded, N024 started, N024 succeeded, N025 started, N025 succeeded, N026 started, N026 succeeded, N027 started, N027 succeeded, N028 started, N028 succeeded, N029 started, N029 succeeded, N030 started, N030 succeeded, N031 started, N031 succeeded, N032 started, N032 succeeded, N033 started, N033 succeeded, N034 started, N034 succeeded, N035 started, N035 succeeded, N036 started, N036 succeeded, N037 started, N037 succeeded, N038 started, N038 succeeded, N039 started, N039 succeeded, N040 started, N040 succeeded, N041 started, N041 succeeded, N042 started, N042 succeeded, N043 started, N043 succeeded, N044 started, N044 succeeded, N045 started, N045 succeeded, N046 started, N046 succeeded, N047 started, N047 succeeded, N048 started, N048 succeeded, N049 started, N049 succeeded, N050 started, N050 succeeded, N051 started, N051 succeeded, N052 started, N052 succeeded, N053 started, N053 succeeded, N054 started, N054 succeeded, N055 started, N055 succeeded, N056 started, N056 succeeded, N057 started, N057 succeeded, N058 started, N058 succeeded, N059 started, N059 succeeded, N060 started, N060 succeeded, N061 started, N061 succeeded, N062 started, N062 succeeded, N063 started, N063 succeeded, N064 started, N064 succeeded, N065 started, N065 succeeded, N066 started, N066 succeeded, N067 started, N067 succeeded, N068 started, N068 succeeded, N069 started, N069 succeeded, N070 started, N070 succeeded, N071 started, N071 succeeded, N072 started, N072 succeeded, N073 started, N073 succeeded, N074 started, N074 succeeded, N075 started, N075 succeeded, N076 started, N076 succeeded, N077 started, N077 succeeded, N078 started, N078 succeeded, N079 started, N079 succeeded, N080 started, N080 succeeded, N081 started, N081 succeeded, N082 started, N082 succeeded, N083 started, N083 succeeded, N084 started, N084 succeeded, N085 started, N085 succeeded, N086 started, N086 succeeded, N087 started, N087 succeeded, N088 started, N088 succeeded, N089 started, N089 succeeded, N090 started, N090 succeeded, N091 started, N091 succeeded, N092 started, N092 succeeded, N093 started, N093 succeeded, N094 started, N094 succeeded, N095 started, N095 succeeded, N096 started, N096 succeeded, N097 started, N097 succeeded, N098 started, N098 succeeded, N099 started, N099 succeeded, N100 started, N100 succeeded, N101 started, N101 succeeded, N102 started, N102 succeeded, N103 started, N103 succeeded, N104 started, N104 succeeded, N105 started, N105 succeeded, N106 started, N106 succeeded, N107 started, N107 succeeded, N108 started, N108 succeeded, N109 started, N109 succeeded, N110 started, N110 succeeded, N111 started, N111 succeeded, N112 started, N112 succeeded, N113 started, N113 succeeded, N114 started, N114 succeeded, N115 started, N115 succeeded, N116 started, N116 succeeded, N117 started, N117 succeeded, N118 started, N118 succeeded, N119 started, N119 succeeded, N120 started, N120 succeeded, N121 started, N121 succeeded, N122 started, N122 succeeded, N123 started, N123 succeeded, N124 started, N124 succeeded, N125 started, N125 succeeded, N126 started, N126 succeeded, N127 started, N127 succeeded, N128 started, N128 succeeded, N129 started, N129 succeeded, N130 started, N130 succeeded, N131 started, N131 succeeded, N132 started, N132 succeeded, N133 started, N133 succeeded, N134 started, N134 succeeded, N135 started, N135 succeeded, N136 started, N136 succeeded, N137 started, N137 succeeded, N138 started, N138 succeeded, N139 started, N139 succeeded, N140 started, N140 succeeded, N141 started, N141 succeeded, N142 started, N142 succeeded, N143 started, N143 succeeded, N144 started, N144 succeeded, N145 started, N145 succeeded, N146 started, N146 succeeded, N147 started, N147 succeeded, N148 started, N148 succeeded, N149 started, N149 succeeded, N150 started, N150 succeeded, N151 started, N151 succeeded, N152 started, N152 succeeded, N153 started, N153 succeeded, N154 started, N154 succeeded, N155 started, N155 succeeded, N156 started, N156 succeeded, N157 started, N157 succeeded, N158 started, N158 succeeded, N159 started, N159 succeeded, N160 started, N160 succeeded, N161 started, N161 succeeded, N162 started, N162 succeeded, N163 started, N163 succeeded, N164 started, N164 succeeded, N165 started, N165 succeeded, N166 started, N166 succeeded, N168 started, N168 succeeded], node_status: {0: Succeeded(String("d7bd2deb-ee27-4661-9e9a-524d734cd6ec")), 1: Succeeded(Object {"auto_restart": Object {"cooldown": Null, "policy": Null}, "boot_disk_id": Null, "hostname": String("the-host"), "identity": Object {"description": String("instance \\"utilization-test-instance\\""), "id": String("d7bd2deb-ee27-4661-9e9a-524d734cd6ec"), "name": String("utilization-test-instance"), "time_created": String("2025-04-30T06:43:54.249576Z"), "time_deleted": Null, "time_modified": String("2025-04-30T06:43:54.249576Z")}, "memory": Number(1073741824), "ncpus": Number(4), "project_id": String("473b39f9-8669-45b3-b3d7-793f7e10fc31"), "runtime_state": Object {"dst_propolis_id": Null, "gen": Number(1), "migration_id": Null, "nexus_state": String("Creating"), "propolis_id": Null, "time_last_auto_restarted": Null, "time_updated": String("2025-04-30T06:43:54.249576Z")}, "updater_gen": Number(1), "updater_id": Null, "user_data": Array [Number(35), Number(99), Number(108), Number(111), Number(117), Number(100), Number(45), Number(99), Number(111), Number(110), Number(102), Number(105), Number(103), Number(10), Number(115), Number(121), Number(115), Number(116), Number(101), Number(109), Number(95), Number(105), Number(110), Number(102), Number(111), Number(58), Number(10), Number(32), Number(32), Number(100), Number(101), Number(102), Number(97), Number(117), Number(108), Number(116), Number(95), Number(117), Number(115), Number(101), Number(114), Number(58), Number(10), Number(32), Number(32), Number(32), Number(32), Number(110), Number(97), Number(109), Number(101), Number(58), Number(32), Number(111), Number(120), Number(105), Number(100), Number(101)]}), 2: Succeeded(Null), 3: Succeeded(Object {"instance_id": String("d7bd2deb-ee27-4661-9e9a-524d734cd6ec"), "new_id": String("64c48731-6cc1-46ff-99f2-86f6c481d0c6"), "saga_params": Object {"boundary_switches": Array [], "create_params": Object {"anti_affinity_groups": Array [], "auto_restart_policy": Null, "boot_disk": Null, "description": String("instance \\"utilization-test-instance\\""), "disks": Array [], "external_ips": Array [], "hostname": String("the-host"), "memory": Number(1073741824), "name": String("utilization-test-instance"), "ncpus": Number(4), "network_interfaces": Object {"type": String("default")}, "ssh_public_keys": Array [], "start": Bool(true), "user_data": String("I2Nsb3VkLWNvbmZpZwpzeXN0ZW1faW5mbzoKICBkZWZhdWx0X3VzZXI6CiAgICBuYW1lOiBveGlkZQ==")}, "project_id": String("473b39f9-8669-45b3-b3d7-793f7e10fc31"), "serialized_authn": Object {"kind": Object {"Authenticated": Array [Object {"actor": Object {"SiloUser": Object {"silo_id": String("001de000-5110-4000-8000-000000000000"), "silo_user_id": String("001de000-05e4-4000-8000-000000004007")}}}, Object {"mapped_fleet_roles": Object {}}]}}}, "which": Number(0)}), 4: Succeeded(Null), 5: Succeeded(Null), 6: Succeeded(Null), 7: Succeeded(Object {"instance_id": String("d7bd2deb-ee27-4661-9e9a-524d734cd6ec"), "new_id": String("79be82ce-957c-439a-b67e-f1dacdb57f1e"), "saga_params": Object {"boundary_switches": Array [], "create_params": Object {"anti_affinity_groups": Array [], "auto_restart_policy": Null, "boot_disk": Null, "description": String("instance \\"utilization-test-instance\\""), "disks": Array [], "external_ips": Array [], "hostname": String("the-host"), "memory": Number(1073741824), "name": String("utilization-test-instance"), "ncpus": Number(4), "network_interfaces": Object {"type": String("default")}, "ssh_public_keys": Array [], "start": Bool(true), "user_data": String("I2Nsb3VkLWNvbmZpZwpzeXN0ZW1faW5mbzoKICBkZWZhdWx0X3VzZXI6CiAgICBuYW1lOiBveGlkZQ==")}, "project_id": String("473b39f9-8669-45b3-b3d7-793f7e10fc31"), "serialized_authn": Object {"kind": Object {"Authenticated": Array [Object {"actor": Object {"SiloUser": Object {"silo_id": String("001de000-5110-4000-8000-000000000000"), "silo_user_id": String("001de000-05e4-4000-8000-000000004007")}}}, Object {"mapped_fleet_roles": Object {}}]}}}, "which": Number(1)}), 8: Succeeded(Null), 9: Succeeded(Null), 10: Succeeded(Null), 11: Succeeded(Object {"instance_id": String("d7bd2deb-ee27-4661-9e9a-524d734cd6ec"), "new_id": String("628c4c7e-b430-4a2a-bfd4-c1a8a5a74fa6"), "saga_params": Object {"boundary_switches": Array [], "create_params": Object {"anti_affinity_groups": Array [], "auto_restart_policy": Null, "boot_disk": Null, "description": String("instance \\"utilization-test-instance\\""), "disks": Array [], "external_ips": Array [], "hostname": String("the-host"), "memory": Number(1073741824), "name": String("utilization-test-instance"), "ncpus": Number(4), "network_interfaces": Object {"type": String("default")}, "ssh_public_keys": Array [], "start": Bool(true), "user_data": String("I2Nsb3VkLWNvbmZpZwpzeXN0ZW1faW5mbzoKICBkZWZhdWx0X3VzZXI6CiAgICBuYW1lOiBveGlkZQ==")}, "project_id": String("473b39f9-8669-45b3-b3d7-793f7e10fc31"), "serialized_authn": Object {"kind": Object {"Authenticated": Array [Object {"actor": Object {"SiloUser": Object {"silo_id": String("001de000-5110-4000-8000-000000000000"), "silo_user_id": String("001de000-05e4-4000-8000-000000004007")}}}, Object {"mapped_fleet_roles": Object {}}]}}}, "which": Number(2)}), 12: Succeeded(Null), 13: Succeeded(Null), 14: Succeeded(Null), 15: Succeeded(Object {"instance_id": String("d7bd2deb-ee27-4661-9e9a-524d734cd6ec"), "new_id": String("9b0302b2-2547-4705-b879-b2d896fa654b"), "saga_params": Object {"boundary_switches": Array [], "create_params": Object {"anti_affinity_groups": Array [], "auto_restart_policy": Null, "boot_disk": Null, "description": String("instance \\"utilization-test-instance\\""), "disks": Array [], "external_ips": Array [], "hostname": String("the-host"), "memory": Number(1073741824), "name": String("utilization-test-instance"), "ncpus": Number(4), "network_interfaces": Object {"type": String("default")}, "ssh_public_keys": Array [], "start": Bool(true), "user_data": String("I2Nsb3VkLWNvbmZpZwpzeXN0ZW1faW5mbzoKICBkZWZhdWx0X3VzZXI6CiAgICBuYW1lOiBveGlkZQ==")}, "project_id": String("473b39f9-8669-45b3-b3d7-793f7e10fc31"), "serialized_authn": Object {"kind": Object {"Authenticated": Array [Object {"actor": Object {"SiloUser": Object {"silo_id": String("001de000-5110-4000-8000-000000000000"), "silo_user_id": String("001de000-05e4-4000-8000-000000004007")}}}, Object {"mapped_fleet_roles": Object {}}]}}}, "which": Number(3)}), 16: Succeeded(Null), 17: Succeeded(Null), 18: Succeeded(Null), 19: Succeeded(Object {"instance_id": String("d7bd2deb-ee27-4661-9e9a-524d734cd6ec"), "new_id": String("7390678b-5d44-43e6-b46d-3c25a5eebd71"), "saga_params": Object {"boundary_switches": Array [], "create_params": Object {"anti_affinity_groups": Array [], "auto_restart_policy": Null, "boot_disk": Null, "description": String("instance \\"utilization-test-instance\\""), "disks": Array [], "external_ips": Array [], "hostname": String("the-host"), "memory": Number(1073741824), "name": String("utilization-test-instance"), "ncpus": Number(4), "network_interfaces": Object {"type": String("default")}, "ssh_public_keys": Array [], "start": Bool(true), "user_data": String("I2Nsb3VkLWNvbmZpZwpzeXN0ZW1faW5mbzoKICBkZWZhdWx0X3VzZXI6CiAgICBuYW1lOiBveGlkZQ==")}, "project_id": String("473b39f9-8669-45b3-b3d7-793f7e10fc31"), "serialized_authn": Object {"kind": Object {"Authenticated": Array [Object {"actor": Object {"SiloUser": Object {"silo_id": String("001de000-5110-4000-8000-000000000000"), "silo_user_id": String("001de000-05e4-4000-8000-000000004007")}}}, Object {"mapped_fleet_roles": Object {}}]}}}, "which": Number(4)}), 20: Succeeded(Null), 21: Succeeded(Null), 22: Succeeded(Null), 23: Succeeded(Object {"instance_id": String("d7bd2deb-ee27-4661-9e9a-524d734cd6ec"), "new_id": String("2a09f6d6-3de1-4bea-b89c-6fc00eaf504e"), "saga_params": Object {"boundary_switches": Array [], "create_params": Object {"anti_affinity_groups": Array [], "auto_restart_policy": Null, "boot_disk": Null, "description": String("instance \\"utilization-test-instance\\""), "disks": Array [], "external_ips": Array [], "hostname": String("the-host"), "memory": Number(1073741824), "name": String("utilization-test-instance"), "ncpus": Number(4), "network_interfaces": Object {"type": String("default")}, "ssh_public_keys": Array [], "start": Bool(true), "user_data": String("I2Nsb3VkLWNvbmZpZwpzeXN0ZW1faW5mbzoKICBkZWZhdWx0X3VzZXI6CiAgICBuYW1lOiBveGlkZQ==")}, "project_id": String("473b39f9-8669-45b3-b3d7-793f7e10fc31"), "serialized_authn": Object {"kind": Object {"Authenticated": Array [Object {"actor": Object {"SiloUser": Object {"silo_id": String("001de000-5110-4000-8000-000000000000"), "silo_user_id": String("001de000-05e4-4000-8000-000000004007")}}}, Object {"mapped_fleet_roles": Object {}}]}}}, "which": Number(5)}), 24: Succeeded(Null), 25: Succeeded(Null), 26: Succeeded(Null), 27: Succeeded(Object {"instance_id": String("d7bd2deb-ee27-4661-9e9a-524d734cd6ec"), "new_id": String("86338488-d577-4ce7-a835-4c7491bc0d88"), "saga_params": Object {"boundary_switches": Array [], "create_params": Object {"anti_affinity_groups": Array [], "auto_restart_policy": Null, "boot_disk": Null, "description": String("instance \\"utilization-test-instance\\""), "disks": Array [], "external_ips": Array [], "hostname": String("the-host"), "memory": Number(1073741824), "name": String("utilization-test-instance"), "ncpus": Number(4), "network_interfaces": Object {"type": String("default")}, "ssh_public_keys": Array [], "start": Bool(true), "user_data": String("I2Nsb3VkLWNvbmZpZwpzeXN0ZW1faW5mbzoKICBkZWZhdWx0X3VzZXI6CiAgICBuYW1lOiBveGlkZQ==")}, "project_id": String("473b39f9-8669-45b3-b3d7-793f7e10fc31"), "serialized_authn": Object {"kind": Object {"Authenticated": Array [Object {"actor": Object {"SiloUser": Object {"silo_id": String("001de000-5110-4000-8000-000000000000"), "silo_user_id": String("001de000-05e4-4000-8000-000000004007")}}}, Object {"mapped_fleet_roles": Object {}}]}}}, "which": Number(6)}), 28: Succeeded(Null), 29: Succeeded(Null), 30: Succeeded(Null), 31: Succeeded(Object {"instance_id": String("d7bd2deb-ee27-4661-9e9a-524d734cd6ec"), "new_id": String("d7156580-5c17-44d4-bbc0-c78d1b1133df"), "saga_params": Object {"boundary_switches": Array [], "create_params": Object {"anti_affinity_groups": Array [], "auto_restart_policy": Null, "boot_disk": Null, "description": String("instance \\"utilization-test-instance\\""), "disks": Array [], "external_ips": Array [], "hostname": String("the-host"), "memory": Number(1073741824), "name": String("utilization-test-instance"), "ncpus": Number(4), "network_interfaces": Object {"type": String("default")}, "ssh_public_keys": Array [], "start": Bool(true), "user_data": String("I2Nsb3VkLWNvbmZpZwpzeXN0ZW1faW5mbzoKICBkZWZhdWx0X3VzZXI6CiAgICBuYW1lOiBveGlkZQ==")}, "project_id": String("473b39f9-8669-45b3-b3d7-793f7e10fc31"), "serialized_authn": Object {"kind": Object {"Authenticated": Array [Object {"actor": Object {"SiloUser": Object {"silo_id": String("001de000-5110-4000-8000-000000000000"), "silo_user_id": String("001de000-05e4-4000-8000-000000004007")}}}, Object {"mapped_fleet_roles": Object {}}]}}}, "which": Number(7)}), 32: Succeeded(Null), 33: Succeeded(Null), 34: Succeeded(Null), 35: Succeeded(String("703ac9cc-cf3f-44f5-9e01-0a385ec2b26a")), 36: Succeeded(Null), 37: Succeeded(Object {"instance_id": String("d7bd2deb-ee27-4661-9e9a-524d734cd6ec"), "new_id": String("d135d564-8cd2-4690-a1c9-1629e2d7c21e"), "saga_params": Object {"boundary_switches": Array [], "create_params": Object {"anti_affinity_groups": Array [], "auto_restart_policy": Null, "boot_disk": Null, "description": String("instance \\"utilization-test-instance\\""), "disks": Array [], "external_ips": Array [], "hostname": String("the-host"), "memory": Number(1073741824), "name": String("utilization-test-instance"), "ncpus": Number(4), "network_interfaces": Object {"type": String("default")}, "ssh_public_keys": Array [], "start": Bool(true), "user_data": String("I2Nsb3VkLWNvbmZpZwpzeXN0ZW1faW5mbzoKICBkZWZhdWx0X3VzZXI6CiAgICBuYW1lOiBveGlkZQ==")}, "project_id": String("473b39f9-8669-45b3-b3d7-793f7e10fc31"), "serialized_authn": Object {"kind": Object {"Authenticated": Array [Object {"actor": Object {"SiloUser": Object {"silo_id": String("001de000-5110-4000-8000-000000000000"), "silo_user_id": String("001de000-05e4-4000-8000-000000004007")}}}, Object {"mapped_fleet_roles": Object {}}]}}}, "which": Number(0)}), 38: Succeeded(Null), 39: Succeeded(Null), 40: Succeeded(Null), 41: Succeeded(Object {"instance_id": String("d7bd2deb-ee27-4661-9e9a-524d734cd6ec"), "new_id": String("60556df4-fdfb-48b6-8878-e6da4a1201e5"), "saga_params": Object {"boundary_switches": Array [], "create_params": Object {"anti_affinity_groups": Array [], "auto_restart_policy": Null, "boot_disk": Null, "description": String("instance \\"utilization-test-instance\\""), "disks": Array [], "external_ips": Array [], "hostname": String("the-host"), "memory": Number(1073741824), "name": String("utilization-test-instance"), "ncpus": Number(4), "network_interfaces": Object {"type": String("default")}, "ssh_public_keys": Array [], "start": Bool(true), "user_data": String("I2Nsb3VkLWNvbmZpZwpzeXN0ZW1faW5mbzoKICBkZWZhdWx0X3VzZXI6CiAgICBuYW1lOiBveGlkZQ==")}, "project_id": String("473b39f9-8669-45b3-b3d7-793f7e10fc31"), "serialized_authn": Object {"kind": Object {"Authenticated": Array [Object {"actor": Object {"SiloUser": Object {"silo_id": String("001de000-5110-4000-8000-000000000000"), "silo_user_id": String("001de000-05e4-4000-8000-000000004007")}}}, Object {"mapped_fleet_roles": Object {}}]}}}, "which": Number(1)}), 42: Succeeded(Null), 43: Succeeded(Null), 44: Succeeded(Null), 45: Succeeded(Object {"instance_id": String("d7bd2deb-ee27-4661-9e9a-524d734cd6ec"), "new_id": String("95e2067e-7259-4904-bcc6-199c85e45b17"), "saga_params": Object {"boundary_switches": Array [], "create_params": Object {"anti_affinity_groups": Array [], "auto_restart_policy": Null, "boot_disk": Null, "description": String("instance \\"utilization-test-instance\\""), "disks": Array [], "external_ips": Array [], "hostname": String("the-host"), "memory": Number(1073741824), "name": String("utilization-test-instance"), "ncpus": Number(4), "network_interfaces": Object {"type": String("default")}, "ssh_public_keys": Array [], "start": Bool(true), "user_data": String("I2Nsb3VkLWNvbmZpZwpzeXN0ZW1faW5mbzoKICBkZWZhdWx0X3VzZXI6CiAgICBuYW1lOiBveGlkZQ==")}, "project_id": String("473b39f9-8669-45b3-b3d7-793f7e10fc31"), "serialized_authn": Object {"kind": Object {"Authenticated": Array [Object {"actor": Object {"SiloUser": Object {"silo_id": String("001de000-5110-4000-8000-000000000000"), "silo_user_id": String("001de000-05e4-4000-8000-000000004007")}}}, Object {"mapped_fleet_roles": Object {}}]}}}, "which": Number(2)}), 46: Succeeded(Null), 47: Succeeded(Null), 48: Succeeded(Null), 49: Succeeded(Object {"instance_id": String("d7bd2deb-ee27-4661-9e9a-524d734cd6ec"), "new_id": String("fdb5b695-fb92-422f-b187-3e8141c4416b"), "saga_params": Object {"boundary_switches": Array [], "create_params": Object {"anti_affinity_groups": Array [], "auto_restart_policy": Null, "boot_disk": Null, "description": String("instance \\"utilization-test-instance\\""), "disks": Array [], "external_ips": Array [], "hostname": String("the-host"), "memory": Number(1073741824), "name": String("utilization-test-instance"), "ncpus": Number(4), "network_interfaces": Object {"type": String("default")}, "ssh_public_keys": Array [], "start": Bool(true), "user_data": String("I2Nsb3VkLWNvbmZpZwpzeXN0ZW1faW5mbzoKICBkZWZhdWx0X3VzZXI6CiAgICBuYW1lOiBveGlkZQ==")}, "project_id": String("473b39f9-8669-45b3-b3d7-793f7e10fc31"), "serialized_authn": Object {"kind": Object {"Authenticated": Array [Object {"actor": Object {"SiloUser": Object {"silo_id": String("001de000-5110-4000-8000-000000000000"), "silo_user_id": String("001de000-05e4-4000-8000-000000004007")}}}, Object {"mapped_fleet_roles": Object {}}]}}}, "which": Number(3)}), 50: Succeeded(Null), 51: Succeeded(Null), 52: Succeeded(Null), 53: Succeeded(Object {"instance_id": String("d7bd2deb-ee27-4661-9e9a-524d734cd6ec"), "new_id": String("1db204ac-45f7-4301-b4bb-8b82e9fce5f5"), "saga_params": Object {"boundary_switches": Array [], "create_params": Object {"anti_affinity_groups": Array [], "auto_restart_policy": Null, "boot_disk": Null, "description": String("instance \\"utilization-test-instance\\""), "disks": Array [], "external_ips": Array [], "hostname": String("the-host"), "memory": Number(1073741824), "name": String("utilization-test-instance"), "ncpus": Number(4), "network_interfaces": Object {"type": String("default")}, "ssh_public_keys": Array [], "start": Bool(true), "user_data": String("I2Nsb3VkLWNvbmZpZwpzeXN0ZW1faW5mbzoKICBkZWZhdWx0X3VzZXI6CiAgICBuYW1lOiBveGlkZQ==")}, "project_id": String("473b39f9-8669-45b3-b3d7-793f7e10fc31"), "serialized_authn": Object {"kind": Object {"Authenticated": Array [Object {"actor": Object {"SiloUser": Object {"silo_id": String("001de000-5110-4000-8000-000000000000"), "silo_user_id": String("001de000-05e4-4000-8000-000000004007")}}}, Object {"mapped_fleet_roles": Object {}}]}}}, "which": Number(4)}), 54: Succeeded(Null), 55: Succeeded(Null), 56: Succeeded(Null), 57: Succeeded(Object {"instance_id": String("d7bd2deb-ee27-4661-9e9a-524d734cd6ec"), "new_id": String("caf1be44-80eb-4e28-91d3-8f81c65992f5"), "saga_params": Object {"boundary_switches": Array [], "create_params": Object {"anti_affinity_groups": Array [], "auto_restart_policy": Null, "boot_disk": Null, "description": String("instance \\"utilization-test-instance\\""), "disks": Array [], "external_ips": Array [], "hostname": String("the-host"), "memory": Number(1073741824), "name": String("utilization-test-instance"), "ncpus": Number(4), "network_interfaces": Object {"type": String("default")}, "ssh_public_keys": Array [], "start": Bool(true), "user_data": String("I2Nsb3VkLWNvbmZpZwpzeXN0ZW1faW5mbzoKICBkZWZhdWx0X3VzZXI6CiAgICBuYW1lOiBveGlkZQ==")}, "project_id": String("473b39f9-8669-45b3-b3d7-793f7e10fc31"), "serialized_authn": Object {"kind": Object {"Authenticated": Array [Object {"actor": Object {"SiloUser": Object {"silo_id": String("001de000-5110-4000-8000-000000000000"), "silo_user_id": String("001de000-05e4-4000-8000-000000004007")}}}, Object {"mapped_fleet_roles": Object {}}]}}}, "which": Number(5)}), 58: Succeeded(Null), 59: Succeeded(Null), 60: Succeeded(Null), 61: Succeeded(Object {"instance_id": String("d7bd2deb-ee27-4661-9e9a-524d734cd6ec"), "new_id": String("c7469230-9dc6-409d-93da-9629008d5d16"), "saga_params": Object {"boundary_switches": Array [], "create_params": Object {"anti_affinity_groups": Array [], "auto_restart_policy": Null, "boot_disk": Null, "description": String("instance \\"utilization-test-instance\\""), "disks": Array [], "external_ips": Array [], "hostname": String("the-host"), "memory": Number(1073741824), "name": String("utilization-test-instance"), "ncpus": Number(4), "network_interfaces": Object {"type": String("default")}, "ssh_public_keys": Array [], "start": Bool(true), "user_data": String("I2Nsb3VkLWNvbmZpZwpzeXN0ZW1faW5mbzoKICBkZWZhdWx0X3VzZXI6CiAgICBuYW1lOiBveGlkZQ==")}, "project_id": String("473b39f9-8669-45b3-b3d7-793f7e10fc31"), "serialized_authn": Object {"kind": Object {"Authenticated": Array [Object {"actor": Object {"SiloUser": Object {"silo_id": String("001de000-5110-4000-8000-000000000000"), "silo_user_id": String("001de000-05e4-4000-8000-000000004007")}}}, Object {"mapped_fleet_roles": Object {}}]}}}, "which": Number(6)}), 62: Succeeded(Null), 63: Succeeded(Null), 64: Succeeded(Null), 65: Succeeded(Object {"instance_id": String("d7bd2deb-ee27-4661-9e9a-524d734cd6ec"), "new_id": String("1943d0ee-055e-4aa1-a7a8-403496b4dbcb"), "saga_params": Object {"boundary_switches": Array [], "create_params": Object {"anti_affinity_groups": Array [], "auto_restart_policy": Null, "boot_disk": Null, "description": String("instance \\"utilization-test-instance\\""), "disks": Array [], "external_ips": Array [], "hostname": String("the-host"), "memory": Number(1073741824), "name": String("utilization-test-instance"), "ncpus": Number(4), "network_interfaces": Object {"type": String("default")}, "ssh_public_keys": Array [], "start": Bool(true), "user_data": String("I2Nsb3VkLWNvbmZpZwpzeXN0ZW1faW5mbzoKICBkZWZhdWx0X3VzZXI6CiAgICBuYW1lOiBveGlkZQ==")}, "project_id": String("473b39f9-8669-45b3-b3d7-793f7e10fc31"), "serialized_authn": Object {"kind": Object {"Authenticated": Array [Object {"actor": Object {"SiloUser": Object {"silo_id": String("001de000-5110-4000-8000-000000000000"), "silo_user_id": String("001de000-05e4-4000-8000-000000004007")}}}, Object {"mapped_fleet_roles": Object {}}]}}}, "which": Number(7)}), 66: Succeeded(Null), 67: Succeeded(Null), 68: Succeeded(Null), 69: Succeeded(Object {"instance_id": String("d7bd2deb-ee27-4661-9e9a-524d734cd6ec"), "new_id": String("e9418c49-b114-4554-82a3-43f65092bf41"), "saga_params": Object {"boundary_switches": Array [], "create_params": Object {"anti_affinity_groups": Array [], "auto_restart_policy": Null, "boot_disk": Null, "description": String("instance \\"utilization-test-instance\\""), "disks": Array [], "external_ips": Array [], "hostname": String("the-host"), "memory": Number(1073741824), "name": String("utilization-test-instance"), "ncpus": Number(4), "network_interfaces": Object {"type": String("default")}, "ssh_public_keys": Array [], "start": Bool(true), "user_data": String("I2Nsb3VkLWNvbmZpZwpzeXN0ZW1faW5mbzoKICBkZWZhdWx0X3VzZXI6CiAgICBuYW1lOiBveGlkZQ==")}, "project_id": String("473b39f9-8669-45b3-b3d7-793f7e10fc31"), "serialized_authn": Object {"kind": Object {"Authenticated": Array [Object {"actor": Object {"SiloUser": Object {"silo_id": String("001de000-5110-4000-8000-000000000000"), "silo_user_id": String("001de000-05e4-4000-8000-000000004007")}}}, Object {"mapped_fleet_roles": Object {}}]}}}, "which": Number(8)}), 70: Succeeded(Null), 71: Succeeded(Null), 72: Succeeded(Null), 73: Succeeded(Object {"instance_id": String("d7bd2deb-ee27-4661-9e9a-524d734cd6ec"), "new_id": String("44dd706b-a60c-43ba-9fee-2f47969c2494"), "saga_params": Object {"boundary_switches": Array [], "create_params": Object {"anti_affinity_groups": Array [], "auto_restart_policy": Null, "boot_disk": Null, "description": String("instance \\"utilization-test-instance\\""), "disks": Array [], "external_ips": Array [], "hostname": String("the-host"), "memory": Number(1073741824), "name": String("utilization-test-instance"), "ncpus": Number(4), "network_interfaces": Object {"type": String("default")}, "ssh_public_keys": Array [], "start": Bool(true), "user_data": String("I2Nsb3VkLWNvbmZpZwpzeXN0ZW1faW5mbzoKICBkZWZhdWx0X3VzZXI6CiAgICBuYW1lOiBveGlkZQ==")}, "project_id": String("473b39f9-8669-45b3-b3d7-793f7e10fc31"), "serialized_authn": Object {"kind": Object {"Authenticated": Array [Object {"actor": Object {"SiloUser": Object {"silo_id": String("001de000-5110-4000-8000-000000000000"), "silo_user_id": String("001de000-05e4-4000-8000-000000004007")}}}, Object {"mapped_fleet_roles": Object {}}]}}}, "which": Number(9)}), 74: Succeeded(Null), 75: Succeeded(Null), 76: Succeeded(Null), 77: Succeeded(Object {"instance_id": String("d7bd2deb-ee27-4661-9e9a-524d734cd6ec"), "new_id": String("22abc147-674c-4b2d-aaaa-f1b4e56bec0d"), "saga_params": Object {"boundary_switches": Array [], "create_params": Object {"anti_affinity_groups": Array [], "auto_restart_policy": Null, "boot_disk": Null, "description": String("instance \\"utilization-test-instance\\""), "disks": Array [], "external_ips": Array [], "hostname": String("the-host"), "memory": Number(1073741824), "name": String("utilization-test-instance"), "ncpus": Number(4), "network_interfaces": Object {"type": String("default")}, "ssh_public_keys": Array [], "start": Bool(true), "user_data": String("I2Nsb3VkLWNvbmZpZwpzeXN0ZW1faW5mbzoKICBkZWZhdWx0X3VzZXI6CiAgICBuYW1lOiBveGlkZQ==")}, "project_id": String("473b39f9-8669-45b3-b3d7-793f7e10fc31"), "serialized_authn": Object {"kind": Object {"Authenticated": Array [Object {"actor": Object {"SiloUser": Object {"silo_id": String("001de000-5110-4000-8000-000000000000"), "silo_user_id": String("001de000-05e4-4000-8000-000000004007")}}}, Object {"mapped_fleet_roles": Object {}}]}}}, "which": Number(10)}), 78: Succeeded(Null), 79: Succeeded(Null), 80: Succeeded(Null), 81: Succeeded(Object {"instance_id": String("d7bd2deb-ee27-4661-9e9a-524d734cd6ec"), "new_id": String("90899943-1eb7-4a82-93b8-3ca4311fc81e"), "saga_params": Object {"boundary_switches": Array [], "create_params": Object {"anti_affinity_groups": Array [], "auto_restart_policy": Null, "boot_disk": Null, "description": String("instance \\"utilization-test-instance\\""), "disks": Array [], "external_ips": Array [], "hostname": String("the-host"), "memory": Number(1073741824), "name": String("utilization-test-instance"), "ncpus": Number(4), "network_interfaces": Object {"type": String("default")}, "ssh_public_keys": Array [], "start": Bool(true), "user_data": String("I2Nsb3VkLWNvbmZpZwpzeXN0ZW1faW5mbzoKICBkZWZhdWx0X3VzZXI6CiAgICBuYW1lOiBveGlkZQ==")}, "project_id": String("473b39f9-8669-45b3-b3d7-793f7e10fc31"), "serialized_authn": Object {"kind": Object {"Authenticated": Array [Object {"actor": Object {"SiloUser": Object {"silo_id": String("001de000-5110-4000-8000-000000000000"), "silo_user_id": String("001de000-05e4-4000-8000-000000004007")}}}, Object {"mapped_fleet_roles": Object {}}]}}}, "which": Number(11)}), 82: Succeeded(Null), 83: Succeeded(Null), 84: Succeeded(Null), 85: Succeeded(Object {"instance_id": String("d7bd2deb-ee27-4661-9e9a-524d734cd6ec"), "new_id": String("6979ef71-cbe0-4065-8900-ae9ef8686f28"), "saga_params": Object {"boundary_switches": Array [], "create_params": Object {"anti_affinity_groups": Array [], "auto_restart_policy": Null, "boot_disk": Null, "description": String("instance \\"utilization-test-instance\\""), "disks": Array [], "external_ips": Array [], "hostname": String("the-host"), "memory": Number(1073741824), "name": String("utilization-test-instance"), "ncpus": Number(4), "network_interfaces": Object {"type": String("default")}, "ssh_public_keys": Array [], "start": Bool(true), "user_data": String("I2Nsb3VkLWNvbmZpZwpzeXN0ZW1faW5mbzoKICBkZWZhdWx0X3VzZXI6CiAgICBuYW1lOiBveGlkZQ==")}, "project_id": String("473b39f9-8669-45b3-b3d7-793f7e10fc31"), "serialized_authn": Object {"kind": Object {"Authenticated": Array [Object {"actor": Object {"SiloUser": Object {"silo_id": String("001de000-5110-4000-8000-000000000000"), "silo_user_id": String("001de000-05e4-4000-8000-000000004007")}}}, Object {"mapped_fleet_roles": Object {}}]}}}, "which": Number(12)}), 86: Succeeded(Null), 87: Succeeded(Null), 88: Succeeded(Null), 89: Succeeded(Object {"instance_id": String("d7bd2deb-ee27-4661-9e9a-524d734cd6ec"), "new_id": String("5e5d8ca1-846b-484c-a5a7-5cc20d5140d3"), "saga_params": Object {"boundary_switches": Array [], "create_params": Object {"anti_affinity_groups": Array [], "auto_restart_policy": Null, "boot_disk": Null, "description": String("instance \\"utilization-test-instance\\""), "disks": Array [], "external_ips": Array [], "hostname": String("the-host"), "memory": Number(1073741824), "name": String("utilization-test-instance"), "ncpus": Number(4), "network_interfaces": Object {"type": String("default")}, "ssh_public_keys": Array [], "start": Bool(true), "user_data": String("I2Nsb3VkLWNvbmZpZwpzeXN0ZW1faW5mbzoKICBkZWZhdWx0X3VzZXI6CiAgICBuYW1lOiBveGlkZQ==")}, "project_id": String("473b39f9-8669-45b3-b3d7-793f7e10fc31"), "serialized_authn": Object {"kind": Object {"Authenticated": Array [Object {"actor": Object {"SiloUser": Object {"silo_id": String("001de000-5110-4000-8000-000000000000"), "silo_user_id": String("001de000-05e4-4000-8000-000000004007")}}}, Object {"mapped_fleet_roles": Object {}}]}}}, "which": Number(13)}), 90: Succeeded(Null), 91: Succeeded(Null), 92: Succeeded(Null), 93: Succeeded(Object {"instance_id": String("d7bd2deb-ee27-4661-9e9a-524d734cd6ec"), "new_id": String("f0b5c727-87b5-41de-aede-b1bc7b951264"), "saga_params": Object {"boundary_switches": Array [], "create_params": Object {"anti_affinity_groups": Array [], "auto_restart_policy": Null, "boot_disk": Null, "description": String("instance \\"utilization-test-instance\\""), "disks": Array [], "external_ips": Array [], "hostname": String("the-host"), "memory": Number(1073741824), "name": String("utilization-test-instance"), "ncpus": Number(4), "network_interfaces": Object {"type": String("default")}, "ssh_public_keys": Array [], "start": Bool(true), "user_data": String("I2Nsb3VkLWNvbmZpZwpzeXN0ZW1faW5mbzoKICBkZWZhdWx0X3VzZXI6CiAgICBuYW1lOiBveGlkZQ==")}, "project_id": String("473b39f9-8669-45b3-b3d7-793f7e10fc31"), "serialized_authn": Object {"kind": Object {"Authenticated": Array [Object {"actor": Object {"SiloUser": Object {"silo_id": String("001de000-5110-4000-8000-000000000000"), "silo_user_id": String("001de000-05e4-4000-8000-000000004007")}}}, Object {"mapped_fleet_roles": Object {}}]}}}, "which": Number(14)}), 94: Succeeded(Null), 95: Succeeded(Null), 96: Succeeded(Null), 97: Succeeded(Object {"instance_id": String("d7bd2deb-ee27-4661-9e9a-524d734cd6ec"), "new_id": String("1c2a0f63-abc4-4eef-b4b2-335cad832ba3"), "saga_params": Object {"boundary_switches": Array [], "create_params": Object {"anti_affinity_groups": Array [], "auto_restart_policy": Null, "boot_disk": Null, "description": String("instance \\"utilization-test-instance\\""), "disks": Array [], "external_ips": Array [], "hostname": String("the-host"), "memory": Number(1073741824), "name": String("utilization-test-instance"), "ncpus": Number(4), "network_interfaces": Object {"type": String("default")}, "ssh_public_keys": Array [], "start": Bool(true), "user_data": String("I2Nsb3VkLWNvbmZpZwpzeXN0ZW1faW5mbzoKICBkZWZhdWx0X3VzZXI6CiAgICBuYW1lOiBveGlkZQ==")}, "project_id": String("473b39f9-8669-45b3-b3d7-793f7e10fc31"), "serialized_authn": Object {"kind": Object {"Authenticated": Array [Object {"actor": Object {"SiloUser": Object {"silo_id": String("001de000-5110-4000-8000-000000000000"), "silo_user_id": String("001de000-05e4-4000-8000-000000004007")}}}, Object {"mapped_fleet_roles": Object {}}]}}}, "which": Number(15)}), 98: Succeeded(Null), 99: Succeeded(Null), 100: Succeeded(Null), 101: Succeeded(Object {"instance_id": String("d7bd2deb-ee27-4661-9e9a-524d734cd6ec"), "new_id": String("f25005d7-f45c-4620-a7ed-94d2eda156e0"), "saga_params": Object {"boundary_switches": Array [], "create_params": Object {"anti_affinity_groups": Array [], "auto_restart_policy": Null, "boot_disk": Null, "description": String("instance \\"utilization-test-instance\\""), "disks": Array [], "external_ips": Array [], "hostname": String("the-host"), "memory": Number(1073741824), "name": String("utilization-test-instance"), "ncpus": Number(4), "network_interfaces": Object {"type": String("default")}, "ssh_public_keys": Array [], "start": Bool(true), "user_data": String("I2Nsb3VkLWNvbmZpZwpzeXN0ZW1faW5mbzoKICBkZWZhdWx0X3VzZXI6CiAgICBuYW1lOiBveGlkZQ==")}, "project_id": String("473b39f9-8669-45b3-b3d7-793f7e10fc31"), "serialized_authn": Object {"kind": Object {"Authenticated": Array [Object {"actor": Object {"SiloUser": Object {"silo_id": String("001de000-5110-4000-8000-000000000000"), "silo_user_id": String("001de000-05e4-4000-8000-000000004007")}}}, Object {"mapped_fleet_roles": Object {}}]}}}, "which": Number(16)}), 102: Succeeded(Null), 103: Succeeded(Null), 104: Succeeded(Null), 105: Succeeded(Object {"instance_id": String("d7bd2deb-ee27-4661-9e9a-524d734cd6ec"), "new_id": String("7340d976-adcc-4ae0-b83b-0dd30bc87c58"), "saga_params": Object {"boundary_switches": Array [], "create_params": Object {"anti_affinity_groups": Array [], "auto_restart_policy": Null, "boot_disk": Null, "description": String("instance \\"utilization-test-instance\\""), "disks": Array [], "external_ips": Array [], "hostname": String("the-host"), "memory": Number(1073741824), "name": String("utilization-test-instance"), "ncpus": Number(4), "network_interfaces": Object {"type": String("default")}, "ssh_public_keys": Array [], "start": Bool(true), "user_data": String("I2Nsb3VkLWNvbmZpZwpzeXN0ZW1faW5mbzoKICBkZWZhdWx0X3VzZXI6CiAgICBuYW1lOiBveGlkZQ==")}, "project_id": String("473b39f9-8669-45b3-b3d7-793f7e10fc31"), "serialized_authn": Object {"kind": Object {"Authenticated": Array [Object {"actor": Object {"SiloUser": Object {"silo_id": String("001de000-5110-4000-8000-000000000000"), "silo_user_id": String("001de000-05e4-4000-8000-000000004007")}}}, Object {"mapped_fleet_roles": Object {}}]}}}, "which": Number(17)}), 106: Succeeded(Null), 107: Succeeded(Null), 108: Succeeded(Null), 109: Succeeded(Object {"instance_id": String("d7bd2deb-ee27-4661-9e9a-524d734cd6ec"), "new_id": String("05e54cff-eed5-453a-98a1-297d8872176d"), "saga_params": Object {"boundary_switches": Array [], "create_params": Object {"anti_affinity_groups": Array [], "auto_restart_policy": Null, "boot_disk": Null, "description": String("instance \\"utilization-test-instance\\""), "disks": Array [], "external_ips": Array [], "hostname": String("the-host"), "memory": Number(1073741824), "name": String("utilization-test-instance"), "ncpus": Number(4), "network_interfaces": Object {"type": String("default")}, "ssh_public_keys": Array [], "start": Bool(true), "user_data": String("I2Nsb3VkLWNvbmZpZwpzeXN0ZW1faW5mbzoKICBkZWZhdWx0X3VzZXI6CiAgICBuYW1lOiBveGlkZQ==")}, "project_id": String("473b39f9-8669-45b3-b3d7-793f7e10fc31"), "serialized_authn": Object {"kind": Object {"Authenticated": Array [Object {"actor": Object {"SiloUser": Object {"silo_id": String("001de000-5110-4000-8000-000000000000"), "silo_user_id": String("001de000-05e4-4000-8000-000000004007")}}}, Object {"mapped_fleet_roles": Object {}}]}}}, "which": Number(18)}), 110: Succeeded(Null), 111: Succeeded(Null), 112: Succeeded(Null), 113: Succeeded(Object {"instance_id": String("d7bd2deb-ee27-4661-9e9a-524d734cd6ec"), "new_id": String("fdb16759-a0d9-45ed-ac3f-ea7a5f13ab24"), "saga_params": Object {"boundary_switches": Array [], "create_params": Object {"anti_affinity_groups": Array [], "auto_restart_policy": Null, "boot_disk": Null, "description": String("instance \\"utilization-test-instance\\""), "disks": Array [], "external_ips": Array [], "hostname": String("the-host"), "memory": Number(1073741824), "name": String("utilization-test-instance"), "ncpus": Number(4), "network_interfaces": Object {"type": String("default")}, "ssh_public_keys": Array [], "start": Bool(true), "user_data": String("I2Nsb3VkLWNvbmZpZwpzeXN0ZW1faW5mbzoKICBkZWZhdWx0X3VzZXI6CiAgICBuYW1lOiBveGlkZQ==")}, "project_id": String("473b39f9-8669-45b3-b3d7-793f7e10fc31"), "serialized_authn": Object {"kind": Object {"Authenticated": Array [Object {"actor": Object {"SiloUser": Object {"silo_id": String("001de000-5110-4000-8000-000000000000"), "silo_user_id": String("001de000-05e4-4000-8000-000000004007")}}}, Object {"mapped_fleet_roles": Object {}}]}}}, "which": Number(19)}), 114: Succeeded(Null), 115: Succeeded(Null), 116: Succeeded(Null), 117: Succeeded(Object {"instance_id": String("d7bd2deb-ee27-4661-9e9a-524d734cd6ec"), "new_id": String("b19a9736-163c-49c2-8115-ee28cc17761a"), "saga_params": Object {"boundary_switches": Array [], "create_params": Object {"anti_affinity_groups": Array [], "auto_restart_policy": Null, "boot_disk": Null, "description": String("instance \\"utilization-test-instance\\""), "disks": Array [], "external_ips": Array [], "hostname": String("the-host"), "memory": Number(1073741824), "name": String("utilization-test-instance"), "ncpus": Number(4), "network_interfaces": Object {"type": String("default")}, "ssh_public_keys": Array [], "start": Bool(true), "user_data": String("I2Nsb3VkLWNvbmZpZwpzeXN0ZW1faW5mbzoKICBkZWZhdWx0X3VzZXI6CiAgICBuYW1lOiBveGlkZQ==")}, "project_id": String("473b39f9-8669-45b3-b3d7-793f7e10fc31"), "serialized_authn": Object {"kind": Object {"Authenticated": Array [Object {"actor": Object {"SiloUser": Object {"silo_id": String("001de000-5110-4000-8000-000000000000"), "silo_user_id": String("001de000-05e4-4000-8000-000000004007")}}}, Object {"mapped_fleet_roles": Object {}}]}}}, "which": Number(20)}), 118: Succeeded(Null), 119: Succeeded(Null), 120: Succeeded(Null), 121: Succeeded(Object {"instance_id": String("d7bd2deb-ee27-4661-9e9a-524d734cd6ec"), "new_id": String("3e95bbf0-31cc-42db-9c13-91a80858663c"), "saga_params": Object {"boundary_switches": Array [], "create_params": Object {"anti_affinity_groups": Array [], "auto_restart_policy": Null, "boot_disk": Null, "description": String("instance \\"utilization-test-instance\\""), "disks": Array [], "external_ips": Array [], "hostname": String("the-host"), "memory": Number(1073741824), "name": String("utilization-test-instance"), "ncpus": Number(4), "network_interfaces": Object {"type": String("default")}, "ssh_public_keys": Array [], "start": Bool(true), "user_data": String("I2Nsb3VkLWNvbmZpZwpzeXN0ZW1faW5mbzoKICBkZWZhdWx0X3VzZXI6CiAgICBuYW1lOiBveGlkZQ==")}, "project_id": String("473b39f9-8669-45b3-b3d7-793f7e10fc31"), "serialized_authn": Object {"kind": Object {"Authenticated": Array [Object {"actor": Object {"SiloUser": Object {"silo_id": String("001de000-5110-4000-8000-000000000000"), "silo_user_id": String("001de000-05e4-4000-8000-000000004007")}}}, Object {"mapped_fleet_roles": Object {}}]}}}, "which": Number(21)}), 122: Succeeded(Null), 123: Succeeded(Null), 124: Succeeded(Null), 125: Succeeded(Object {"instance_id": String("d7bd2deb-ee27-4661-9e9a-524d734cd6ec"), "new_id": String("a4cb59af-13e0-43d4-9e6a-cb804af95f94"), "saga_params": Object {"boundary_switches": Array [], "create_params": Object {"anti_affinity_groups": Array [], "auto_restart_policy": Null, "boot_disk": Null, "description": String("instance \\"utilization-test-instance\\""), "disks": Array [], "external_ips": Array [], "hostname": String("the-host"), "memory": Number(1073741824), "name": String("utilization-test-instance"), "ncpus": Number(4), "network_interfaces": Object {"type": String("default")}, "ssh_public_keys": Array [], "start": Bool(true), "user_data": String("I2Nsb3VkLWNvbmZpZwpzeXN0ZW1faW5mbzoKICBkZWZhdWx0X3VzZXI6CiAgICBuYW1lOiBveGlkZQ==")}, "project_id": String("473b39f9-8669-45b3-b3d7-793f7e10fc31"), "serialized_authn": Object {"kind": Object {"Authenticated": Array [Object {"actor": Object {"SiloUser": Object {"silo_id": String("001de000-5110-4000-8000-000000000000"), "silo_user_id": String("001de000-05e4-4000-8000-000000004007")}}}, Object {"mapped_fleet_roles": Object {}}]}}}, "which": Number(22)}), 126: Succeeded(Null), 127: Succeeded(Null), 128: Succeeded(Null), 129: Succeeded(Object {"instance_id": String("d7bd2deb-ee27-4661-9e9a-524d734cd6ec"), "new_id": String("f4bf029b-6ff1-4269-a3aa-78663047295b"), "saga_params": Object {"boundary_switches": Array [], "create_params": Object {"anti_affinity_groups": Array [], "auto_restart_policy": Null, "boot_disk": Null, "description": String("instance \\"utilization-test-instance\\""), "disks": Array [], "external_ips": Array [], "hostname": String("the-host"), "memory": Number(1073741824), "name": String("utilization-test-instance"), "ncpus": Number(4), "network_interfaces": Object {"type": String("default")}, "ssh_public_keys": Array [], "start": Bool(true), "user_data": String("I2Nsb3VkLWNvbmZpZwpzeXN0ZW1faW5mbzoKICBkZWZhdWx0X3VzZXI6CiAgICBuYW1lOiBveGlkZQ==")}, "project_id": String("473b39f9-8669-45b3-b3d7-793f7e10fc31"), "serialized_authn": Object {"kind": Object {"Authenticated": Array [Object {"actor": Object {"SiloUser": Object {"silo_id": String("001de000-5110-4000-8000-000000000000"), "silo_user_id": String("001de000-05e4-4000-8000-000000004007")}}}, Object {"mapped_fleet_roles": Object {}}]}}}, "which": Number(23)}), 130: Succeeded(Null), 131: Succeeded(Null), 132: Succeeded(Null), 133: Succeeded(Object {"instance_id": String("d7bd2deb-ee27-4661-9e9a-524d734cd6ec"), "new_id": String("34abc739-27ab-4357-820a-4bd83d55c78b"), "saga_params": Object {"boundary_switches": Array [], "create_params": Object {"anti_affinity_groups": Array [], "auto_restart_policy": Null, "boot_disk": Null, "description": String("instance \\"utilization-test-instance\\""), "disks": Array [], "external_ips": Array [], "hostname": String("the-host"), "memory": Number(1073741824), "name": String("utilization-test-instance"), "ncpus": Number(4), "network_interfaces": Object {"type": String("default")}, "ssh_public_keys": Array [], "start": Bool(true), "user_data": String("I2Nsb3VkLWNvbmZpZwpzeXN0ZW1faW5mbzoKICBkZWZhdWx0X3VzZXI6CiAgICBuYW1lOiBveGlkZQ==")}, "project_id": String("473b39f9-8669-45b3-b3d7-793f7e10fc31"), "serialized_authn": Object {"kind": Object {"Authenticated": Array [Object {"actor": Object {"SiloUser": Object {"silo_id": String("001de000-5110-4000-8000-000000000000"), "silo_user_id": String("001de000-05e4-4000-8000-000000004007")}}}, Object {"mapped_fleet_roles": Object {}}]}}}, "which": Number(24)}), 134: Succeeded(Null), 135: Succeeded(Null), 136: Succeeded(Null), 137: Succeeded(Object {"instance_id": String("d7bd2deb-ee27-4661-9e9a-524d734cd6ec"), "new_id": String("c4c3995e-045b-4152-ad96-930a44eb1c91"), "saga_params": Object {"boundary_switches": Array [], "create_params": Object {"anti_affinity_groups": Array [], "auto_restart_policy": Null, "boot_disk": Null, "description": String("instance \\"utilization-test-instance\\""), "disks": Array [], "external_ips": Array [], "hostname": String("the-host"), "memory": Number(1073741824), "name": String("utilization-test-instance"), "ncpus": Number(4), "network_interfaces": Object {"type": String("default")}, "ssh_public_keys": Array [], "start": Bool(true), "user_data": String("I2Nsb3VkLWNvbmZpZwpzeXN0ZW1faW5mbzoKICBkZWZhdWx0X3VzZXI6CiAgICBuYW1lOiBveGlkZQ==")}, "project_id": String("473b39f9-8669-45b3-b3d7-793f7e10fc31"), "serialized_authn": Object {"kind": Object {"Authenticated": Array [Object {"actor": Object {"SiloUser": Object {"silo_id": String("001de000-5110-4000-8000-000000000000"), "silo_user_id": String("001de000-05e4-4000-8000-000000004007")}}}, Object {"mapped_fleet_roles": Object {}}]}}}, "which": Number(25)}), 138: Succeeded(Null), 139: Succeeded(Null), 140: Succeeded(Null), 141: Succeeded(Object {"instance_id": String("d7bd2deb-ee27-4661-9e9a-524d734cd6ec"), "new_id": String("aa70062c-595c-4b83-8d78-1efc9c2f5978"), "saga_params": Object {"boundary_switches": Array [], "create_params": Object {"anti_affinity_groups": Array [], "auto_restart_policy": Null, "boot_disk": Null, "description": String("instance \\"utilization-test-instance\\""), "disks": Array [], "external_ips": Array [], "hostname": String("the-host"), "memory": Number(1073741824), "name": String("utilization-test-instance"), "ncpus": Number(4), "network_interfaces": Object {"type": String("default")}, "ssh_public_keys": Array [], "start": Bool(true), "user_data": String("I2Nsb3VkLWNvbmZpZwpzeXN0ZW1faW5mbzoKICBkZWZhdWx0X3VzZXI6CiAgICBuYW1lOiBveGlkZQ==")}, "project_id": String("473b39f9-8669-45b3-b3d7-793f7e10fc31"), "serialized_authn": Object {"kind": Object {"Authenticated": Array [Object {"actor": Object {"SiloUser": Object {"silo_id": String("001de000-5110-4000-8000-000000000000"), "silo_user_id": String("001de000-05e4-4000-8000-000000004007")}}}, Object {"mapped_fleet_roles": Object {}}]}}}, "which": Number(26)}), 142: Succeeded(Null), 143: Succeeded(Null), 144: Succeeded(Null), 145: Succeeded(Object {"instance_id": String("d7bd2deb-ee27-4661-9e9a-524d734cd6ec"), "new_id": String("25bbbf93-a75b-4a07-b175-ad5ff6de72e6"), "saga_params": Object {"boundary_switches": Array [], "create_params": Object {"anti_affinity_groups": Array [], "auto_restart_policy": Null, "boot_disk": Null, "description": String("instance \\"utilization-test-instance\\""), "disks": Array [], "external_ips": Array [], "hostname": String("the-host"), "memory": Number(1073741824), "name": String("utilization-test-instance"), "ncpus": Number(4), "network_interfaces": Object {"type": String("default")}, "ssh_public_keys": Array [], "start": Bool(true), "user_data": String("I2Nsb3VkLWNvbmZpZwpzeXN0ZW1faW5mbzoKICBkZWZhdWx0X3VzZXI6CiAgICBuYW1lOiBveGlkZQ==")}, "project_id": String("473b39f9-8669-45b3-b3d7-793f7e10fc31"), "serialized_authn": Object {"kind": Object {"Authenticated": Array [Object {"actor": Object {"SiloUser": Object {"silo_id": String("001de000-5110-4000-8000-000000000000"), "silo_user_id": String("001de000-05e4-4000-8000-000000004007")}}}, Object {"mapped_fleet_roles": Object {}}]}}}, "which": Number(27)}), 146: Succeeded(Null), 147: Succeeded(Null), 148: Succeeded(Null), 149: Succeeded(Object {"instance_id": String("d7bd2deb-ee27-4661-9e9a-524d734cd6ec"), "new_id": String("6047d897-d5be-45b8-93ff-0710079dfeae"), "saga_params": Object {"boundary_switches": Array [], "create_params": Object {"anti_affinity_groups": Array [], "auto_restart_policy": Null, "boot_disk": Null, "description": String("instance \\"utilization-test-instance\\""), "disks": Array [], "external_ips": Array [], "hostname": String("the-host"), "memory": Number(1073741824), "name": String("utilization-test-instance"), "ncpus": Number(4), "network_interfaces": Object {"type": String("default")}, "ssh_public_keys": Array [], "start": Bool(true), "user_data": String("I2Nsb3VkLWNvbmZpZwpzeXN0ZW1faW5mbzoKICBkZWZhdWx0X3VzZXI6CiAgICBuYW1lOiBveGlkZQ==")}, "project_id": String("473b39f9-8669-45b3-b3d7-793f7e10fc31"), "serialized_authn": Object {"kind": Object {"Authenticated": Array [Object {"actor": Object {"SiloUser": Object {"silo_id": String("001de000-5110-4000-8000-000000000000"), "silo_user_id": String("001de000-05e4-4000-8000-000000004007")}}}, Object {"mapped_fleet_roles": Object {}}]}}}, "which": Number(28)}), 150: Succeeded(Null), 151: Succeeded(Null), 152: Succeeded(Null), 153: Succeeded(Object {"instance_id": String("d7bd2deb-ee27-4661-9e9a-524d734cd6ec"), "new_id": String("fe58941b-b766-487e-812e-f5e79533a730"), "saga_params": Object {"boundary_switches": Array [], "create_params": Object {"anti_affinity_groups": Array [], "auto_restart_policy": Null, "boot_disk": Null, "description": String("instance \\"utilization-test-instance\\""), "disks": Array [], "external_ips": Array [], "hostname": String("the-host"), "memory": Number(1073741824), "name": String("utilization-test-instance"), "ncpus": Number(4), "network_interfaces": Object {"type": String("default")}, "ssh_public_keys": Array [], "start": Bool(true), "user_data": String("I2Nsb3VkLWNvbmZpZwpzeXN0ZW1faW5mbzoKICBkZWZhdWx0X3VzZXI6CiAgICBuYW1lOiBveGlkZQ==")}, "project_id": String("473b39f9-8669-45b3-b3d7-793f7e10fc31"), "serialized_authn": Object {"kind": Object {"Authenticated": Array [Object {"actor": Object {"SiloUser": Object {"silo_id": String("001de000-5110-4000-8000-000000000000"), "silo_user_id": String("001de000-05e4-4000-8000-000000004007")}}}, Object {"mapped_fleet_roles": Object {}}]}}}, "which": Number(29)}), 154: Succeeded(Null), 155: Succeeded(Null), 156: Succeeded(Null), 157: Succeeded(Object {"instance_id": String("d7bd2deb-ee27-4661-9e9a-524d734cd6ec"), "new_id": String("d4aeb710-e637-4ddd-b35b-dd02d5495d12"), "saga_params": Object {"boundary_switches": Array [], "create_params": Object {"anti_affinity_groups": Array [], "auto_restart_policy": Null, "boot_disk": Null, "description": String("instance \\"utilization-test-instance\\""), "disks": Array [], "external_ips": Array [], "hostname": String("the-host"), "memory": Number(1073741824), "name": String("utilization-test-instance"), "ncpus": Number(4), "network_interfaces": Object {"type": String("default")}, "ssh_public_keys": Array [], "start": Bool(true), "user_data": String("I2Nsb3VkLWNvbmZpZwpzeXN0ZW1faW5mbzoKICBkZWZhdWx0X3VzZXI6CiAgICBuYW1lOiBveGlkZQ==")}, "project_id": String("473b39f9-8669-45b3-b3d7-793f7e10fc31"), "serialized_authn": Object {"kind": Object {"Authenticated": Array [Object {"actor": Object {"SiloUser": Object {"silo_id": String("001de000-5110-4000-8000-000000000000"), "silo_user_id": String("001de000-05e4-4000-8000-000000004007")}}}, Object {"mapped_fleet_roles": Object {}}]}}}, "which": Number(30)}), 158: Succeeded(Null), 159: Succeeded(Null), 160: Succeeded(Null), 161: Succeeded(Object {"instance_id": String("d7bd2deb-ee27-4661-9e9a-524d734cd6ec"), "new_id": String("28acecea-be1f-451f-ac0d-066b7436b4d2"), "saga_params": Object {"boundary_switches": Array [], "create_params": Object {"anti_affinity_groups": Array [], "auto_restart_policy": Null, "boot_disk": Null, "description": String("instance \\"utilization-test-instance\\""), "disks": Array [], "external_ips": Array [], "hostname": String("the-host"), "memory": Number(1073741824), "name": String("utilization-test-instance"), "ncpus": Number(4), "network_interfaces": Object {"type": String("default")}, "ssh_public_keys": Array [], "start": Bool(true), "user_data": String("I2Nsb3VkLWNvbmZpZwpzeXN0ZW1faW5mbzoKICBkZWZhdWx0X3VzZXI6CiAgICBuYW1lOiBveGlkZQ==")}, "project_id": String("473b39f9-8669-45b3-b3d7-793f7e10fc31"), "serialized_authn": Object {"kind": Object {"Authenticated": Array [Object {"actor": Object {"SiloUser": Object {"silo_id": String("001de000-5110-4000-8000-000000000000"), "silo_user_id": String("001de000-05e4-4000-8000-000000004007")}}}, Object {"mapped_fleet_roles": Object {}}]}}}, "which": Number(31)}), 162: Succeeded(Null), 163: Succeeded(Null), 164: Succeeded(Null), 165: Succeeded(Null), 166: Succeeded(Null), 167: Succeeded(Null), 168: Succeeded(Null)} }, kind: Ok(SagaResultOk { saga_output: Null, node_outputs: {"external-ip-0": Null, "external-ip-1": Null, "external-ip-10": Null, "external-ip-11": Null, "external-ip-12": Null, "external-ip-13": Null, "external-ip-14": Null, "external-ip-15": Null, "external-ip-16": Null, "external-ip-17": Null, "external-ip-18": Null, "external-ip-19": Null, "external-ip-2": Null, "external-ip-20": Null, "external-ip-21": Null, "external-ip-22": Null, "external-ip-23": Null, "external-ip-24": Null, "external-ip-25": Null, "external-ip-26": Null, "external-ip-27": Null, "external-ip-28": Null, "external-ip-29": Null, "external-ip-3": Null, "external-ip-30": Null, "external-ip-31": Null, "external-ip-4": Null, "external-ip-5": Null, "external-ip-6": Null, "external-ip-7": Null, "external-ip-8": Null, "external-ip-9": Null, "external_ip0": Null, "external_ip1": Null, "external_ip10": Null, "external_ip11": Null, "external_ip12": Null, "external_ip13": Null, "external_ip14": Null, "external_ip15": Null, "external_ip16": Null, "external_ip17": Null, "external_ip18": Null, "external_ip19": Null, "external_ip2": Null, "external_ip20": Null, "external_ip21": Null, "external_ip22": Null, "external_ip23": Null, "external_ip24": Null, "external_ip25": Null, "external_ip26": Null, "external_ip27": Null, "external_ip28": Null, "external_ip29": Null, "external_ip3": Null, "external_ip30": Null, "external_ip31": Null, "external_ip4": Null, "external_ip5": Null, "external_ip6": Null, "external_ip7": Null, "external_ip8": Null, "external_ip9": Null, "external_ip_params0": Object {"instance_id": String("d7bd2deb-ee27-4661-9e9a-524d734cd6ec"), "new_id": String("d135d564-8cd2-4690-a1c9-1629e2d7c21e"), "saga_params": Object {"boundary_switches": Array [], "create_params": Object {"anti_affinity_groups": Array [], "auto_restart_policy": Null, "boot_disk": Null, "description": String("instance \\"utilization-test-instance\\""), "disks": Array [], "external_ips": Array [], "hostname": String("the-host"), "memory": Number(1073741824), "name": String("utilization-test-instance"), "ncpus": Number(4), "network_interfaces": Object {"type": String("default")}, "ssh_public_keys": Array [], "start": Bool(true), "user_data": String("I2Nsb3VkLWNvbmZpZwpzeXN0ZW1faW5mbzoKICBkZWZhdWx0X3VzZXI6CiAgICBuYW1lOiBveGlkZQ==")}, "project_id": String("473b39f9-8669-45b3-b3d7-793f7e10fc31"), "serialized_authn": Object {"kind": Object {"Authenticated": Array [Object {"actor": Object {"SiloUser": Object {"silo_id": String("001de000-5110-4000-8000-000000000000"), "silo_user_id": String("001de000-05e4-4000-8000-000000004007")}}}, Object {"mapped_fleet_roles": Object {}}]}}}, "which": Number(0)}, "external_ip_params1": Object {"instance_id": String("d7bd2deb-ee27-4661-9e9a-524d734cd6ec"), "new_id": String("60556df4-fdfb-48b6-8878-e6da4a1201e5"), "saga_params": Object {"boundary_switches": Array [], "create_params": Object {"anti_affinity_groups": Array [], "auto_restart_policy": Null, "boot_disk": Null, "description": String("instance \\"utilization-test-instance\\""), "disks": Array [], "external_ips": Array [], "hostname": String("the-host"), "memory": Number(1073741824), "name": String("utilization-test-instance"), "ncpus": Number(4), "network_interfaces": Object {"type": String("default")}, "ssh_public_keys": Array [], "start": Bool(true), "user_data": String("I2Nsb3VkLWNvbmZpZwpzeXN0ZW1faW5mbzoKICBkZWZhdWx0X3VzZXI6CiAgICBuYW1lOiBveGlkZQ==")}, "project_id": String("473b39f9-8669-45b3-b3d7-793f7e10fc31"), "serialized_authn": Object {"kind": Object {"Authenticated": Array [Object {"actor": Object {"SiloUser": Object {"silo_id": String("001de000-5110-4000-8000-000000000000"), "silo_user_id": String("001de000-05e4-4000-8000-000000004007")}}}, Object {"mapped_fleet_roles": Object {}}]}}}, "which": Number(1)}, "external_ip_params10": Object {"instance_id": String("d7bd2deb-ee27-4661-9e9a-524d734cd6ec"), "new_id": String("22abc147-674c-4b2d-aaaa-f1b4e56bec0d"), "saga_params": Object {"boundary_switches": Array [], "create_params": Object {"anti_affinity_groups": Array [], "auto_restart_policy": Null, "boot_disk": Null, "description": String("instance \\"utilization-test-instance\\""), "disks": Array [], "external_ips": Array [], "hostname": String("the-host"), "memory": Number(1073741824), "name": String("utilization-test-instance"), "ncpus": Number(4), "network_interfaces": Object {"type": String("default")}, "ssh_public_keys": Array [], "start": Bool(true), "user_data": String("I2Nsb3VkLWNvbmZpZwpzeXN0ZW1faW5mbzoKICBkZWZhdWx0X3VzZXI6CiAgICBuYW1lOiBveGlkZQ==")}, "project_id": String("473b39f9-8669-45b3-b3d7-793f7e10fc31"), "serialized_authn": Object {"kind": Object {"Authenticated": Array [Object {"actor": Object {"SiloUser": Object {"silo_id": String("001de000-5110-4000-8000-000000000000"), "silo_user_id": String("001de000-05e4-4000-8000-000000004007")}}}, Object {"mapped_fleet_roles": Object {}}]}}}, "which": Number(10)}, "external_ip_params11": Object {"instance_id": String("d7bd2deb-ee27-4661-9e9a-524d734cd6ec"), "new_id": String("90899943-1eb7-4a82-93b8-3ca4311fc81e"), "saga_params": Object {"boundary_switches": Array [], "create_params": Object {"anti_affinity_groups": Array [], "auto_restart_policy": Null, "boot_disk": Null, "description": String("instance \\"utilization-test-instance\\""), "disks": Array [], "external_ips": Array [], "hostname": String("the-host"), "memory": Number(1073741824), "name": String("utilization-test-instance"), "ncpus": Number(4), "network_interfaces": Object {"type": String("default")}, "ssh_public_keys": Array [], "start": Bool(true), "user_data": String("I2Nsb3VkLWNvbmZpZwpzeXN0ZW1faW5mbzoKICBkZWZhdWx0X3VzZXI6CiAgICBuYW1lOiBveGlkZQ==")}, "project_id": String("473b39f9-8669-45b3-b3d7-793f7e10fc31"), "serialized_authn": Object {"kind": Object {"Authenticated": Array [Object {"actor": Object {"SiloUser": Object {"silo_id": String("001de000-5110-4000-8000-000000000000"), "silo_user_id": String("001de000-05e4-4000-8000-000000004007")}}}, Object {"mapped_fleet_roles": Object {}}]}}}, "which": Number(11)}, "external_ip_params12": Object {"instance_id": String("d7bd2deb-ee27-4661-9e9a-524d734cd6ec"), "new_id": String("6979ef71-cbe0-4065-8900-ae9ef8686f28"), "saga_params": Object {"boundary_switches": Array [], "create_params": Object {"anti_affinity_groups": Array [], "auto_restart_policy": Null, "boot_disk": Null, "description": String("instance \\"utilization-test-instance\\""), "disks": Array [], "external_ips": Array [], "hostname": String("the-host"), "memory": Number(1073741824), "name": String("utilization-test-instance"), "ncpus": Number(4), "network_interfaces": Object {"type": String("default")}, "ssh_public_keys": Array [], "start": Bool(true), "user_data": String("I2Nsb3VkLWNvbmZpZwpzeXN0ZW1faW5mbzoKICBkZWZhdWx0X3VzZXI6CiAgICBuYW1lOiBveGlkZQ==")}, "project_id": String("473b39f9-8669-45b3-b3d7-793f7e10fc31"), "serialized_authn": Object {"kind": Object {"Authenticated": Array [Object {"actor": Object {"SiloUser": Object {"silo_id": String("001de000-5110-4000-8000-000000000000"), "silo_user_id": String("001de000-05e4-4000-8000-000000004007")}}}, Object {"mapped_fleet_roles": Object {}}]}}}, "which": Number(12)}, "external_ip_params13": Object {"instance_id": String("d7bd2deb-ee27-4661-9e9a-524d734cd6ec"), "new_id": String("5e5d8ca1-846b-484c-a5a7-5cc20d5140d3"), "saga_params": Object {"boundary_switches": Array [], "create_params": Object {"anti_affinity_groups": Array [], "auto_restart_policy": Null, "boot_disk": Null, "description": String("instance \\"utilization-test-instance\\""), "disks": Array [], "external_ips": Array [], "hostname": String("the-host"), "memory": Number(1073741824), "name": String("utilization-test-instance"), "ncpus": Number(4), "network_interfaces": Object {"type": String("default")}, "ssh_public_keys": Array [], "start": Bool(true), "user_data": String("I2Nsb3VkLWNvbmZpZwpzeXN0ZW1faW5mbzoKICBkZWZhdWx0X3VzZXI6CiAgICBuYW1lOiBveGlkZQ==")}, "project_id": String("473b39f9-8669-45b3-b3d7-793f7e10fc31"), "serialized_authn": Object {"kind": Object {"Authenticated": Array [Object {"actor": Object {"SiloUser": Object {"silo_id": String("001de000-5110-4000-8000-000000000000"), "silo_user_id": String("001de000-05e4-4000-8000-000000004007")}}}, Object {"mapped_fleet_roles": Object {}}]}}}, "which": Number(13)}, "external_ip_params14": Object {"instance_id": String("d7bd2deb-ee27-4661-9e9a-524d734cd6ec"), "new_id": String("f0b5c727-87b5-41de-aede-b1bc7b951264"), "saga_params": Object {"boundary_switches": Array [], "create_params": Object {"anti_affinity_groups": Array [], "auto_restart_policy": Null, "boot_disk": Null, "description": String("instance \\"utilization-test-instance\\""), "disks": Array [], "external_ips": Array [], "hostname": String("the-host"), "memory": Number(1073741824), "name": String("utilization-test-instance"), "ncpus": Number(4), "network_interfaces": Object {"type": String("default")}, "ssh_public_keys": Array [], "start": Bool(true), "user_data": String("I2Nsb3VkLWNvbmZpZwpzeXN0ZW1faW5mbzoKICBkZWZhdWx0X3VzZXI6CiAgICBuYW1lOiBveGlkZQ==")}, "project_id": String("473b39f9-8669-45b3-b3d7-793f7e10fc31"), "serialized_authn": Object {"kind": Object {"Authenticated": Array [Object {"actor": Object {"SiloUser": Object {"silo_id": String("001de000-5110-4000-8000-000000000000"), "silo_user_id": String("001de000-05e4-4000-8000-000000004007")}}}, Object {"mapped_fleet_roles": Object {}}]}}}, "which": Number(14)}, "external_ip_params15": Object {"instance_id": String("d7bd2deb-ee27-4661-9e9a-524d734cd6ec"), "new_id": String("1c2a0f63-abc4-4eef-b4b2-335cad832ba3"), "saga_params": Object {"boundary_switches": Array [], "create_params": Object {"anti_affinity_groups": Array [], "auto_restart_policy": Null, "boot_disk": Null, "description": String("instance \\"utilization-test-instance\\""), "disks": Array [], "external_ips": Array [], "hostname": String("the-host"), "memory": Number(1073741824), "name": String("utilization-test-instance"), "ncpus": Number(4), "network_interfaces": Object {"type": String("default")}, "ssh_public_keys": Array [], "start": Bool(true), "user_data": String("I2Nsb3VkLWNvbmZpZwpzeXN0ZW1faW5mbzoKICBkZWZhdWx0X3VzZXI6CiAgICBuYW1lOiBveGlkZQ==")}, "project_id": String("473b39f9-8669-45b3-b3d7-793f7e10fc31"), "serialized_authn": Object {"kind": Object {"Authenticated": Array [Object {"actor": Object {"SiloUser": Object {"silo_id": String("001de000-5110-4000-8000-000000000000"), "silo_user_id": String("001de000-05e4-4000-8000-000000004007")}}}, Object {"mapped_fleet_roles": Object {}}]}}}, "which": Number(15)}, "external_ip_params16": Object {"instance_id": String("d7bd2deb-ee27-4661-9e9a-524d734cd6ec"), "new_id": String("f25005d7-f45c-4620-a7ed-94d2eda156e0"), "saga_params": Object {"boundary_switches": Array [], "create_params": Object {"anti_affinity_groups": Array [], "auto_restart_policy": Null, "boot_disk": Null, "description": String("instance \\"utilization-test-instance\\""), "disks": Array [], "external_ips": Array [], "hostname": String("the-host"), "memory": Number(1073741824), "name": String("utilization-test-instance"), "ncpus": Number(4), "network_interfaces": Object {"type": String("default")}, "ssh_public_keys": Array [], "start": Bool(true), "user_data": String("I2Nsb3VkLWNvbmZpZwpzeXN0ZW1faW5mbzoKICBkZWZhdWx0X3VzZXI6CiAgICBuYW1lOiBveGlkZQ==")}, "project_id": String("473b39f9-8669-45b3-b3d7-793f7e10fc31"), "serialized_authn": Object {"kind": Object {"Authenticated": Array [Object {"actor": Object {"SiloUser": Object {"silo_id": String("001de000-5110-4000-8000-000000000000"), "silo_user_id": String("001de000-05e4-4000-8000-000000004007")}}}, Object {"mapped_fleet_roles": Object {}}]}}}, "which": Number(16)}, "external_ip_params17": Object {"instance_id": String("d7bd2deb-ee27-4661-9e9a-524d734cd6ec"), "new_id": String("7340d976-adcc-4ae0-b83b-0dd30bc87c58"), "saga_params": Object {"boundary_switches": Array [], "create_params": Object {"anti_affinity_groups": Array [], "auto_restart_policy": Null, "boot_disk": Null, "description": String("instance \\"utilization-test-instance\\""), "disks": Array [], "external_ips": Array [], "hostname": String("the-host"), "memory": Number(1073741824), "name": String("utilization-test-instance"), "ncpus": Number(4), "network_interfaces": Object {"type": String("default")}, "ssh_public_keys": Array [], "start": Bool(true), "user_data": String("I2Nsb3VkLWNvbmZpZwpzeXN0ZW1faW5mbzoKICBkZWZhdWx0X3VzZXI6CiAgICBuYW1lOiBveGlkZQ==")}, "project_id": String("473b39f9-8669-45b3-b3d7-793f7e10fc31"), "serialized_authn": Object {"kind": Object {"Authenticated": Array [Object {"actor": Object {"SiloUser": Object {"silo_id": String("001de000-5110-4000-8000-000000000000"), "silo_user_id": String("001de000-05e4-4000-8000-000000004007")}}}, Object {"mapped_fleet_roles": Object {}}]}}}, "which": Number(17)}, "external_ip_params18": Object {"instance_id": String("d7bd2deb-ee27-4661-9e9a-524d734cd6ec"), "new_id": String("05e54cff-eed5-453a-98a1-297d8872176d"), "saga_params": Object {"boundary_switches": Array [], "create_params": Object {"anti_affinity_groups": Array [], "auto_restart_policy": Null, "boot_disk": Null, "description": String("instance \\"utilization-test-instance\\""), "disks": Array [], "external_ips": Array [], "hostname": String("the-host"), "memory": Number(1073741824), "name": String("utilization-test-instance"), "ncpus": Number(4), "network_interfaces": Object {"type": String("default")}, "ssh_public_keys": Array [], "start": Bool(true), "user_data": String("I2Nsb3VkLWNvbmZpZwpzeXN0ZW1faW5mbzoKICBkZWZhdWx0X3VzZXI6CiAgICBuYW1lOiBveGlkZQ==")}, "project_id": String("473b39f9-8669-45b3-b3d7-793f7e10fc31"), "serialized_authn": Object {"kind": Object {"Authenticated": Array [Object {"actor": Object {"SiloUser": Object {"silo_id": String("001de000-5110-4000-8000-000000000000"), "silo_user_id": String("001de000-05e4-4000-8000-000000004007")}}}, Object {"mapped_fleet_roles": Object {}}]}}}, "which": Number(18)}, "external_ip_params19": Object {"instance_id": String("d7bd2deb-ee27-4661-9e9a-524d734cd6ec"), "new_id": String("fdb16759-a0d9-45ed-ac3f-ea7a5f13ab24"), "saga_params": Object {"boundary_switches": Array [], "create_params": Object {"anti_affinity_groups": Array [], "auto_restart_policy": Null, "boot_disk": Null, "description": String("instance \\"utilization-test-instance\\""), "disks": Array [], "external_ips": Array [], "hostname": String("the-host"), "memory": Number(1073741824), "name": String("utilization-test-instance"), "ncpus": Number(4), "network_interfaces": Object {"type": String("default")}, "ssh_public_keys": Array [], "start": Bool(true), "user_data": String("I2Nsb3VkLWNvbmZpZwpzeXN0ZW1faW5mbzoKICBkZWZhdWx0X3VzZXI6CiAgICBuYW1lOiBveGlkZQ==")}, "project_id": String("473b39f9-8669-45b3-b3d7-793f7e10fc31"), "serialized_authn": Object {"kind": Object {"Authenticated": Array [Object {"actor": Object {"SiloUser": Object {"silo_id": String("001de000-5110-4000-8000-000000000000"), "silo_user_id": String("001de000-05e4-4000-8000-000000004007")}}}, Object {"mapped_fleet_roles": Object {}}]}}}, "which": Number(19)}, "external_ip_params2": Object {"instance_id": String("d7bd2deb-ee27-4661-9e9a-524d734cd6ec"), "new_id": String("95e2067e-7259-4904-bcc6-199c85e45b17"), "saga_params": Object {"boundary_switches": Array [], "create_params": Object {"anti_affinity_groups": Array [], "auto_restart_policy": Null, "boot_disk": Null, "description": String("instance \\"utilization-test-instance\\""), "disks": Array [], "external_ips": Array [], "hostname": String("the-host"), "memory": Number(1073741824), "name": String("utilization-test-instance"), "ncpus": Number(4), "network_interfaces": Object {"type": String("default")}, "ssh_public_keys": Array [], "start": Bool(true), "user_data": String("I2Nsb3VkLWNvbmZpZwpzeXN0ZW1faW5mbzoKICBkZWZhdWx0X3VzZXI6CiAgICBuYW1lOiBveGlkZQ==")}, "project_id": String("473b39f9-8669-45b3-b3d7-793f7e10fc31"), "serialized_authn": Object {"kind": Object {"Authenticated": Array [Object {"actor": Object {"SiloUser": Object {"silo_id": String("001de000-5110-4000-8000-000000000000"), "silo_user_id": String("001de000-05e4-4000-8000-000000004007")}}}, Object {"mapped_fleet_roles": Object {}}]}}}, "which": Number(2)}, "external_ip_params20": Object {"instance_id": String("d7bd2deb-ee27-4661-9e9a-524d734cd6ec"), "new_id": String("b19a9736-163c-49c2-8115-ee28cc17761a"), "saga_params": Object {"boundary_switches": Array [], "create_params": Object {"anti_affinity_groups": Array [], "auto_restart_policy": Null, "boot_disk": Null, "description": String("instance \\"utilization-test-instance\\""), "disks": Array [], "external_ips": Array [], "hostname": String("the-host"), "memory": Number(1073741824), "name": String("utilization-test-instance"), "ncpus": Number(4), "network_interfaces": Object {"type": String("default")}, "ssh_public_keys": Array [], "start": Bool(true), "user_data": String("I2Nsb3VkLWNvbmZpZwpzeXN0ZW1faW5mbzoKICBkZWZhdWx0X3VzZXI6CiAgICBuYW1lOiBveGlkZQ==")}, "project_id": String("473b39f9-8669-45b3-b3d7-793f7e10fc31"), "serialized_authn": Object {"kind": Object {"Authenticated": Array [Object {"actor": Object {"SiloUser": Object {"silo_id": String("001de000-5110-4000-8000-000000000000"), "silo_user_id": String("001de000-05e4-4000-8000-000000004007")}}}, Object {"mapped_fleet_roles": Object {}}]}}}, "which": Number(20)}, "external_ip_params21": Object {"instance_id": String("d7bd2deb-ee27-4661-9e9a-524d734cd6ec"), "new_id": String("3e95bbf0-31cc-42db-9c13-91a80858663c"), "saga_params": Object {"boundary_switches": Array [], "create_params": Object {"anti_affinity_groups": Array [], "auto_restart_policy": Null, "boot_disk": Null, "description": String("instance \\"utilization-test-instance\\""), "disks": Array [], "external_ips": Array [], "hostname": String("the-host"), "memory": Number(1073741824), "name": String("utilization-test-instance"), "ncpus": Number(4), "network_interfaces": Object {"type": String("default")}, "ssh_public_keys": Array [], "start": Bool(true), "user_data": String("I2Nsb3VkLWNvbmZpZwpzeXN0ZW1faW5mbzoKICBkZWZhdWx0X3VzZXI6CiAgICBuYW1lOiBveGlkZQ==")}, "project_id": String("473b39f9-8669-45b3-b3d7-793f7e10fc31"), "serialized_authn": Object {"kind": Object {"Authenticated": Array [Object {"actor": Object {"SiloUser": Object {"silo_id": String("001de000-5110-4000-8000-000000000000"), "silo_user_id": String("001de000-05e4-4000-8000-000000004007")}}}, Object {"mapped_fleet_roles": Object {}}]}}}, "which": Number(21)}, "external_ip_params22": Object {"instance_id": String("d7bd2deb-ee27-4661-9e9a-524d734cd6ec"), "new_id": String("a4cb59af-13e0-43d4-9e6a-cb804af95f94"), "saga_params": Object {"boundary_switches": Array [], "create_params": Object {"anti_affinity_groups": Array [], "auto_restart_policy": Null, "boot_disk": Null, "description": String("instance \\"utilization-test-instance\\""), "disks": Array [], "external_ips": Array [], "hostname": String("the-host"), "memory": Number(1073741824), "name": String("utilization-test-instance"), "ncpus": Number(4), "network_interfaces": Object {"type": String("default")}, "ssh_public_keys": Array [], "start": Bool(true), "user_data": String("I2Nsb3VkLWNvbmZpZwpzeXN0ZW1faW5mbzoKICBkZWZhdWx0X3VzZXI6CiAgICBuYW1lOiBveGlkZQ==")}, "project_id": String("473b39f9-8669-45b3-b3d7-793f7e10fc31"), "serialized_authn": Object {"kind": Object {"Authenticated": Array [Object {"actor": Object {"SiloUser": Object {"silo_id": String("001de000-5110-4000-8000-000000000000"), "silo_user_id": String("001de000-05e4-4000-8000-000000004007")}}}, Object {"mapped_fleet_roles": Object {}}]}}}, "which": Number(22)}, "external_ip_params23": Object {"instance_id": String("d7bd2deb-ee27-4661-9e9a-524d734cd6ec"), "new_id": String("f4bf029b-6ff1-4269-a3aa-78663047295b"), "saga_params": Object {"boundary_switches": Array [], "create_params": Object {"anti_affinity_groups": Array [], "auto_restart_policy": Null, "boot_disk": Null, "description": String("instance \\"utilization-test-instance\\""), "disks": Array [], "external_ips": Array [], "hostname": String("the-host"), "memory": Number(1073741824), "name": String("utilization-test-instance"), "ncpus": Number(4), "network_interfaces": Object {"type": String("default")}, "ssh_public_keys": Array [], "start": Bool(true), "user_data": String("I2Nsb3VkLWNvbmZpZwpzeXN0ZW1faW5mbzoKICBkZWZhdWx0X3VzZXI6CiAgICBuYW1lOiBveGlkZQ==")}, "project_id": String("473b39f9-8669-45b3-b3d7-793f7e10fc31"), "serialized_authn": Object {"kind": Object {"Authenticated": Array [Object {"actor": Object {"SiloUser": Object {"silo_id": String("001de000-5110-4000-8000-000000000000"), "silo_user_id": String("001de000-05e4-4000-8000-000000004007")}}}, Object {"mapped_fleet_roles": Object {}}]}}}, "which": Number(23)}, "external_ip_params24": Object {"instance_id": String("d7bd2deb-ee27-4661-9e9a-524d734cd6ec"), "new_id": String("34abc739-27ab-4357-820a-4bd83d55c78b"), "saga_params": Object {"boundary_switches": Array [], "create_params": Object {"anti_affinity_groups": Array [], "auto_restart_policy": Null, "boot_disk": Null, "description": String("instance \\"utilization-test-instance\\""), "disks": Array [], "external_ips": Array [], "hostname": String("the-host"), "memory": Number(1073741824), "name": String("utilization-test-instance"), "ncpus": Number(4), "network_interfaces": Object {"type": String("default")}, "ssh_public_keys": Array [], "start": Bool(true), "user_data": String("I2Nsb3VkLWNvbmZpZwpzeXN0ZW1faW5mbzoKICBkZWZhdWx0X3VzZXI6CiAgICBuYW1lOiBveGlkZQ==")}, "project_id": String("473b39f9-8669-45b3-b3d7-793f7e10fc31"), "serialized_authn": Object {"kind": Object {"Authenticated": Array [Object {"actor": Object {"SiloUser": Object {"silo_id": String("001de000-5110-4000-8000-000000000000"), "silo_user_id": String("001de000-05e4-4000-8000-000000004007")}}}, Object {"mapped_fleet_roles": Object {}}]}}}, "which": Number(24)}, "external_ip_params25": Object {"instance_id": String("d7bd2deb-ee27-4661-9e9a-524d734cd6ec"), "new_id": String("c4c3995e-045b-4152-ad96-930a44eb1c91"), "saga_params": Object {"boundary_switches": Array [], "create_params": Object {"anti_affinity_groups": Array [], "auto_restart_policy": Null, "boot_disk": Null, "description": String("instance \\"utilization-test-instance\\""), "disks": Array [], "external_ips": Array [], "hostname": String("the-host"), "memory": Number(1073741824), "name": String("utilization-test-instance"), "ncpus": Number(4), "network_interfaces": Object {"type": String("default")}, "ssh_public_keys": Array [], "start": Bool(true), "user_data": String("I2Nsb3VkLWNvbmZpZwpzeXN0ZW1faW5mbzoKICBkZWZhdWx0X3VzZXI6CiAgICBuYW1lOiBveGlkZQ==")}, "project_id": String("473b39f9-8669-45b3-b3d7-793f7e10fc31"), "serialized_authn": Object {"kind": Object {"Authenticated": Array [Object {"actor": Object {"SiloUser": Object {"silo_id": String("001de000-5110-4000-8000-000000000000"), "silo_user_id": String("001de000-05e4-4000-8000-000000004007")}}}, Object {"mapped_fleet_roles": Object {}}]}}}, "which": Number(25)}, "external_ip_params26": Object {"instance_id": String("d7bd2deb-ee27-4661-9e9a-524d734cd6ec"), "new_id": String("aa70062c-595c-4b83-8d78-1efc9c2f5978"), "saga_params": Object {"boundary_switches": Array [], "create_params": Object {"anti_affinity_groups": Array [], "auto_restart_policy": Null, "boot_disk": Null, "description": String("instance \\"utilization-test-instance\\""), "disks": Array [], "external_ips": Array [], "hostname": String("the-host"), "memory": Number(1073741824), "name": String("utilization-test-instance"), "ncpus": Number(4), "network_interfaces": Object {"type": String("default")}, "ssh_public_keys": Array [], "start": Bool(true), "user_data": String("I2Nsb3VkLWNvbmZpZwpzeXN0ZW1faW5mbzoKICBkZWZhdWx0X3VzZXI6CiAgICBuYW1lOiBveGlkZQ==")}, "project_id": String("473b39f9-8669-45b3-b3d7-793f7e10fc31"), "serialized_authn": Object {"kind": Object {"Authenticated": Array [Object {"actor": Object {"SiloUser": Object {"silo_id": String("001de000-5110-4000-8000-000000000000"), "silo_user_id": String("001de000-05e4-4000-8000-000000004007")}}}, Object {"mapped_fleet_roles": Object {}}]}}}, "which": Number(26)}, "external_ip_params27": Object {"instance_id": String("d7bd2deb-ee27-4661-9e9a-524d734cd6ec"), "new_id": String("25bbbf93-a75b-4a07-b175-ad5ff6de72e6"), "saga_params": Object {"boundary_switches": Array [], "create_params": Object {"anti_affinity_groups": Array [], "auto_restart_policy": Null, "boot_disk": Null, "description": String("instance \\"utilization-test-instance\\""), "disks": Array [], "external_ips": Array [], "hostname": String("the-host"), "memory": Number(1073741824), "name": String("utilization-test-instance"), "ncpus": Number(4), "network_interfaces": Object {"type": String("default")}, "ssh_public_keys": Array [], "start": Bool(true), "user_data": String("I2Nsb3VkLWNvbmZpZwpzeXN0ZW1faW5mbzoKICBkZWZhdWx0X3VzZXI6CiAgICBuYW1lOiBveGlkZQ==")}, "project_id": String("473b39f9-8669-45b3-b3d7-793f7e10fc31"), "serialized_authn": Object {"kind": Object {"Authenticated": Array [Object {"actor": Object {"SiloUser": Object {"silo_id": String("001de000-5110-4000-8000-000000000000"), "silo_user_id": String("001de000-05e4-4000-8000-000000004007")}}}, Object {"mapped_fleet_roles": Object {}}]}}}, "which": Number(27)}, "external_ip_params28": Object {"instance_id": String("d7bd2deb-ee27-4661-9e9a-524d734cd6ec"), "new_id": String("6047d897-d5be-45b8-93ff-0710079dfeae"), "saga_params": Object {"boundary_switches": Array [], "create_params": Object {"anti_affinity_groups": Array [], "auto_restart_policy": Null, "boot_disk": Null, "description": String("instance \\"utilization-test-instance\\""), "disks": Array [], "external_ips": Array [], "hostname": String("the-host"), "memory": Number(1073741824), "name": String("utilization-test-instance"), "ncpus": Number(4), "network_interfaces": Object {"type": String("default")}, "ssh_public_keys": Array [], "start": Bool(true), "user_data": String("I2Nsb3VkLWNvbmZpZwpzeXN0ZW1faW5mbzoKICBkZWZhdWx0X3VzZXI6CiAgICBuYW1lOiBveGlkZQ==")}, "project_id": String("473b39f9-8669-45b3-b3d7-793f7e10fc31"), "serialized_authn": Object {"kind": Object {"Authenticated": Array [Object {"actor": Object {"SiloUser": Object {"silo_id": String("001de000-5110-4000-8000-000000000000"), "silo_user_id": String("001de000-05e4-4000-8000-000000004007")}}}, Object {"mapped_fleet_roles": Object {}}]}}}, "which": Number(28)}, "external_ip_params29": Object {"instance_id": String("d7bd2deb-ee27-4661-9e9a-524d734cd6ec"), "new_id": String("fe58941b-b766-487e-812e-f5e79533a730"), "saga_params": Object {"boundary_switches": Array [], "create_params": Object {"anti_affinity_groups": Array [], "auto_restart_policy": Null, "boot_disk": Null, "description": String("instance \\"utilization-test-instance\\""), "disks": Array [], "external_ips": Array [], "hostname": String("the-host"), "memory": Number(1073741824), "name": String("utilization-test-instance"), "ncpus": Number(4), "network_interfaces": Object {"type": String("default")}, "ssh_public_keys": Array [], "start": Bool(true), "user_data": String("I2Nsb3VkLWNvbmZpZwpzeXN0ZW1faW5mbzoKICBkZWZhdWx0X3VzZXI6CiAgICBuYW1lOiBveGlkZQ==")}, "project_id": String("473b39f9-8669-45b3-b3d7-793f7e10fc31"), "serialized_authn": Object {"kind": Object {"Authenticated": Array [Object {"actor": Object {"SiloUser": Object {"silo_id": String("001de000-5110-4000-8000-000000000000"), "silo_user_id": String("001de000-05e4-4000-8000-000000004007")}}}, Object {"mapped_fleet_roles": Object {}}]}}}, "which": Number(29)}, "external_ip_params3": Object {"instance_id": String("d7bd2deb-ee27-4661-9e9a-524d734cd6ec"), "new_id": String("fdb5b695-fb92-422f-b187-3e8141c4416b"), "saga_params": Object {"boundary_switches": Array [], "create_params": Object {"anti_affinity_groups": Array [], "auto_restart_policy": Null, "boot_disk": Null, "description": String("instance \\"utilization-test-instance\\""), "disks": Array [], "external_ips": Array [], "hostname": String("the-host"), "memory": Number(1073741824), "name": String("utilization-test-instance"), "ncpus": Number(4), "network_interfaces": Object {"type": String("default")}, "ssh_public_keys": Array [], "start": Bool(true), "user_data": String("I2Nsb3VkLWNvbmZpZwpzeXN0ZW1faW5mbzoKICBkZWZhdWx0X3VzZXI6CiAgICBuYW1lOiBveGlkZQ==")}, "project_id": String("473b39f9-8669-45b3-b3d7-793f7e10fc31"), "serialized_authn": Object {"kind": Object {"Authenticated": Array [Object {"actor": Object {"SiloUser": Object {"silo_id": String("001de000-5110-4000-8000-000000000000"), "silo_user_id": String("001de000-05e4-4000-8000-000000004007")}}}, Object {"mapped_fleet_roles": Object {}}]}}}, "which": Number(3)}, "external_ip_params30": Object {"instance_id": String("d7bd2deb-ee27-4661-9e9a-524d734cd6ec"), "new_id": String("d4aeb710-e637-4ddd-b35b-dd02d5495d12"), "saga_params": Object {"boundary_switches": Array [], "create_params": Object {"anti_affinity_groups": Array [], "auto_restart_policy": Null, "boot_disk": Null, "description": String("instance \\"utilization-test-instance\\""), "disks": Array [], "external_ips": Array [], "hostname": String("the-host"), "memory": Number(1073741824), "name": String("utilization-test-instance"), "ncpus": Number(4), "network_interfaces": Object {"type": String("default")}, "ssh_public_keys": Array [], "start": Bool(true), "user_data": String("I2Nsb3VkLWNvbmZpZwpzeXN0ZW1faW5mbzoKICBkZWZhdWx0X3VzZXI6CiAgICBuYW1lOiBveGlkZQ==")}, "project_id": String("473b39f9-8669-45b3-b3d7-793f7e10fc31"), "serialized_authn": Object {"kind": Object {"Authenticated": Array [Object {"actor": Object {"SiloUser": Object {"silo_id": String("001de000-5110-4000-8000-000000000000"), "silo_user_id": String("001de000-05e4-4000-8000-000000004007")}}}, Object {"mapped_fleet_roles": Object {}}]}}}, "which": Number(30)}, "external_ip_params31": Object {"instance_id": String("d7bd2deb-ee27-4661-9e9a-524d734cd6ec"), "new_id": String("28acecea-be1f-451f-ac0d-066b7436b4d2"), "saga_params": Object {"boundary_switches": Array [], "create_params": Object {"anti_affinity_groups": Array [], "auto_restart_policy": Null, "boot_disk": Null, "description": String("instance \\"utilization-test-instance\\""), "disks": Array [], "external_ips": Array [], "hostname": String("the-host"), "memory": Number(1073741824), "name": String("utilization-test-instance"), "ncpus": Number(4), "network_interfaces": Object {"type": String("default")}, "ssh_public_keys": Array [], "start": Bool(true), "user_data": String("I2Nsb3VkLWNvbmZpZwpzeXN0ZW1faW5mbzoKICBkZWZhdWx0X3VzZXI6CiAgICBuYW1lOiBveGlkZQ==")}, "project_id": String("473b39f9-8669-45b3-b3d7-793f7e10fc31"), "serialized_authn": Object {"kind": Object {"Authenticated": Array [Object {"actor": Object {"SiloUser": Object {"silo_id": String("001de000-5110-4000-8000-000000000000"), "silo_user_id": String("001de000-05e4-4000-8000-000000004007")}}}, Object {"mapped_fleet_roles": Object {}}]}}}, "which": Number(31)}, "external_ip_params4": Object {"instance_id": String("d7bd2deb-ee27-4661-9e9a-524d734cd6ec"), "new_id": String("1db204ac-45f7-4301-b4bb-8b82e9fce5f5"), "saga_params": Object {"boundary_switches": Array [], "create_params": Object {"anti_affinity_groups": Array [], "auto_restart_policy": Null, "boot_disk": Null, "description": String("instance \\"utilization-test-instance\\""), "disks": Array [], "external_ips": Array [], "hostname": String("the-host"), "memory": Number(1073741824), "name": String("utilization-test-instance"), "ncpus": Number(4), "network_interfaces": Object {"type": String("default")}, "ssh_public_keys": Array [], "start": Bool(true), "user_data": String("I2Nsb3VkLWNvbmZpZwpzeXN0ZW1faW5mbzoKICBkZWZhdWx0X3VzZXI6CiAgICBuYW1lOiBveGlkZQ==")}, "project_id": String("473b39f9-8669-45b3-b3d7-793f7e10fc31"), "serialized_authn": Object {"kind": Object {"Authenticated": Array [Object {"actor": Object {"SiloUser": Object {"silo_id": String("001de000-5110-4000-8000-000000000000"), "silo_user_id": String("001de000-05e4-4000-8000-000000004007")}}}, Object {"mapped_fleet_roles": Object {}}]}}}, "which": Number(4)}, "external_ip_params5": Object {"instance_id": String("d7bd2deb-ee27-4661-9e9a-524d734cd6ec"), "new_id": String("caf1be44-80eb-4e28-91d3-8f81c65992f5"), "saga_params": Object {"boundary_switches": Array [], "create_params": Object {"anti_affinity_groups": Array [], "auto_restart_policy": Null, "boot_disk": Null, "description": String("instance \\"utilization-test-instance\\""), "disks": Array [], "external_ips": Array [], "hostname": String("the-host"), "memory": Number(1073741824), "name": String("utilization-test-instance"), "ncpus": Number(4), "network_interfaces": Object {"type": String("default")}, "ssh_public_keys": Array [], "start": Bool(true), "user_data": String("I2Nsb3VkLWNvbmZpZwpzeXN0ZW1faW5mbzoKICBkZWZhdWx0X3VzZXI6CiAgICBuYW1lOiBveGlkZQ==")}, "project_id": String("473b39f9-8669-45b3-b3d7-793f7e10fc31"), "serialized_authn": Object {"kind": Object {"Authenticated": Array [Object {"actor": Object {"SiloUser": Object {"silo_id": String("001de000-5110-4000-8000-000000000000"), "silo_user_id": String("001de000-05e4-4000-8000-000000004007")}}}, Object {"mapped_fleet_roles": Object {}}]}}}, "which": Number(5)}, "external_ip_params6": Object {"instance_id": String("d7bd2deb-ee27-4661-9e9a-524d734cd6ec"), "new_id": String("c7469230-9dc6-409d-93da-9629008d5d16"), "saga_params": Object {"boundary_switches": Array [], "create_params": Object {"anti_affinity_groups": Array [], "auto_restart_policy": Null, "boot_disk": Null, "description": String("instance \\"utilization-test-instance\\""), "disks": Array [], "external_ips": Array [], "hostname": String("the-host"), "memory": Number(1073741824), "name": String("utilization-test-instance"), "ncpus": Number(4), "network_interfaces": Object {"type": String("default")}, "ssh_public_keys": Array [], "start": Bool(true), "user_data": String("I2Nsb3VkLWNvbmZpZwpzeXN0ZW1faW5mbzoKICBkZWZhdWx0X3VzZXI6CiAgICBuYW1lOiBveGlkZQ==")}, "project_id": String("473b39f9-8669-45b3-b3d7-793f7e10fc31"), "serialized_authn": Object {"kind": Object {"Authenticated": Array [Object {"actor": Object {"SiloUser": Object {"silo_id": String("001de000-5110-4000-8000-000000000000"), "silo_user_id": String("001de000-05e4-4000-8000-000000004007")}}}, Object {"mapped_fleet_roles": Object {}}]}}}, "which": Number(6)}, "external_ip_params7": Object {"instance_id": String("d7bd2deb-ee27-4661-9e9a-524d734cd6ec"), "new_id": String("1943d0ee-055e-4aa1-a7a8-403496b4dbcb"), "saga_params": Object {"boundary_switches": Array [], "create_params": Object {"anti_affinity_groups": Array [], "auto_restart_policy": Null, "boot_disk": Null, "description": String("instance \\"utilization-test-instance\\""), "disks": Array [], "external_ips": Array [], "hostname": String("the-host"), "memory": Number(1073741824), "name": String("utilization-test-instance"), "ncpus": Number(4), "network_interfaces": Object {"type": String("default")}, "ssh_public_keys": Array [], "start": Bool(true), "user_data": String("I2Nsb3VkLWNvbmZpZwpzeXN0ZW1faW5mbzoKICBkZWZhdWx0X3VzZXI6CiAgICBuYW1lOiBveGlkZQ==")}, "project_id": String("473b39f9-8669-45b3-b3d7-793f7e10fc31"), "serialized_authn": Object {"kind": Object {"Authenticated": Array [Object {"actor": Object {"SiloUser": Object {"silo_id": String("001de000-5110-4000-8000-000000000000"), "silo_user_id": String("001de000-05e4-4000-8000-000000004007")}}}, Object {"mapped_fleet_roles": Object {}}]}}}, "which": Number(7)}, "external_ip_params8": Object {"instance_id": String("d7bd2deb-ee27-4661-9e9a-524d734cd6ec"), "new_id": String("e9418c49-b114-4554-82a3-43f65092bf41"), "saga_params": Object {"boundary_switches": Array [], "create_params": Object {"anti_affinity_groups": Array [], "auto_restart_policy": Null, "boot_disk": Null, "description": String("instance \\"utilization-test-instance\\""), "disks": Array [], "external_ips": Array [], "hostname": String("the-host"), "memory": Number(1073741824), "name": String("utilization-test-instance"), "ncpus": Number(4), "network_interfaces": Object {"type": String("default")}, "ssh_public_keys": Array [], "start": Bool(true), "user_data": String("I2Nsb3VkLWNvbmZpZwpzeXN0ZW1faW5mbzoKICBkZWZhdWx0X3VzZXI6CiAgICBuYW1lOiBveGlkZQ==")}, "project_id": String("473b39f9-8669-45b3-b3d7-793f7e10fc31"), "serialized_authn": Object {"kind": Object {"Authenticated": Array [Object {"actor": Object {"SiloUser": Object {"silo_id": String("001de000-5110-4000-8000-000000000000"), "silo_user_id": String("001de000-05e4-4000-8000-000000004007")}}}, Object {"mapped_fleet_roles": Object {}}]}}}, "which": Number(8)}, "external_ip_params9": Object {"instance_id": String("d7bd2deb-ee27-4661-9e9a-524d734cd6ec"), "new_id": String("44dd706b-a60c-43ba-9fee-2f47969c2494"), "saga_params": Object {"boundary_switches": Array [], "create_params": Object {"anti_affinity_groups": Array [], "auto_restart_policy": Null, "boot_disk": Null, "description": String("instance \\"utilization-test-instance\\""), "disks": Array [], "external_ips": Array [], "hostname": String("the-host"), "memory": Number(1073741824), "name": String("utilization-test-instance"), "ncpus": Number(4), "network_interfaces": Object {"type": String("default")}, "ssh_public_keys": Array [], "start": Bool(true), "user_data": String("I2Nsb3VkLWNvbmZpZwpzeXN0ZW1faW5mbzoKICBkZWZhdWx0X3VzZXI6CiAgICBuYW1lOiBveGlkZQ==")}, "project_id": String("473b39f9-8669-45b3-b3d7-793f7e10fc31"), "serialized_authn": Object {"kind": Object {"Authenticated": Array [Object {"actor": Object {"SiloUser": Object {"silo_id": String("001de000-5110-4000-8000-000000000000"), "silo_user_id": String("001de000-05e4-4000-8000-000000004007")}}}, Object {"mapped_fleet_roles": Object {}}]}}}, "which": Number(9)}, "instance_id": String("d7bd2deb-ee27-4661-9e9a-524d734cd6ec"), "instance_record": Object {"auto_restart": Object {"cooldown": Null, "policy": Null}, "boot_disk_id": Null, "hostname": String("the-host"), "identity": Object {"description": String("instance \\"utilization-test-instance\\""), "id": String("d7bd2deb-ee27-4661-9e9a-524d734cd6ec"), "name": String("utilization-test-instance"), "time_created": String("2025-04-30T06:43:54.249576Z"), "time_deleted": Null, "time_modified": String("2025-04-30T06:43:54.249576Z")}, "memory": Number(1073741824), "ncpus": Number(4), "project_id": String("473b39f9-8669-45b3-b3d7-793f7e10fc31"), "runtime_state": Object {"dst_propolis_id": Null, "gen": Number(1), "migration_id": Null, "nexus_state": String("Creating"), "propolis_id": Null, "time_last_auto_restarted": Null, "time_updated": String("2025-04-30T06:43:54.249576Z")}, "updater_gen": Number(1), "updater_id": Null, "user_data": Array [Number(35), Number(99), Number(108), Number(111), Number(117), Number(100), Number(45), Number(99), Number(111), Number(110), Number(102), Number(105), Number(103), Number(10), Number(115), Number(121), Number(115), Number(116), Number(101), Number(109), Number(95), Number(105), Number(110), Number(102), Number(111), Number(58), Number(10), Number(32), Number(32), Number(100), Number(101), Number(102), Number(97), Number(117), Number(108), Number(116), Number(95), Number(117), Number(115), Number(101), Number(114), Number(58), Number(10), Number(32), Number(32), Number(32), Number(32), Number(110), Number(97), Number(109), Number(101), Number(58), Number(32), Number(111), Number(120), Number(105), Number(100), Number(101)]}, "network_interface0": Null, "network_interface1": Null, "network_interface2": Null, "network_interface3": Null, "network_interface4": Null, "network_interface5": Null, "network_interface6": Null, "network_interface7": Null, "network_interface_params0": Object {"instance_id": String("d7bd2deb-ee27-4661-9e9a-524d734cd6ec"), "new_id": String("64c48731-6cc1-46ff-99f2-86f6c481d0c6"), "saga_params": Object {"boundary_switches": Array [], "create_params": Object {"anti_affinity_groups": Array [], "auto_restart_policy": Null, "boot_disk": Null, "description": String("instance \\"utilization-test-instance\\""), "disks": Array [], "external_ips": Array [], "hostname": String("the-host"), "memory": Number(1073741824), "name": String("utilization-test-instance"), "ncpus": Number(4), "network_interfaces": Object {"type": String("default")}, "ssh_public_keys": Array [], "start": Bool(true), "user_data": String("I2Nsb3VkLWNvbmZpZwpzeXN0ZW1faW5mbzoKICBkZWZhdWx0X3VzZXI6CiAgICBuYW1lOiBveGlkZQ==")}, "project_id": String("473b39f9-8669-45b3-b3d7-793f7e10fc31"), "serialized_authn": Object {"kind": Object {"Authenticated": Array [Object {"actor": Object {"SiloUser": Object {"silo_id": String("001de000-5110-4000-8000-000000000000"), "silo_user_id": String("001de000-05e4-4000-8000-000000004007")}}}, Object {"mapped_fleet_roles": Object {}}]}}}, "which": Number(0)}, "network_interface_params1": Object {"instance_id": String("d7bd2deb-ee27-4661-9e9a-524d734cd6ec"), "new_id": String("79be82ce-957c-439a-b67e-f1dacdb57f1e"), "saga_params": Object {"boundary_switches": Array [], "create_params": Object {"anti_affinity_groups": Array [], "auto_restart_policy": Null, "boot_disk": Null, "description": String("instance \\"utilization-test-instance\\""), "disks": Array [], "external_ips": Array [], "hostname": String("the-host"), "memory": Number(1073741824), "name": String("utilization-test-instance"), "ncpus": Number(4), "network_interfaces": Object {"type": String("default")}, "ssh_public_keys": Array [], "start": Bool(true), "user_data": String("I2Nsb3VkLWNvbmZpZwpzeXN0ZW1faW5mbzoKICBkZWZhdWx0X3VzZXI6CiAgICBuYW1lOiBveGlkZQ==")}, "project_id": String("473b39f9-8669-45b3-b3d7-793f7e10fc31"), "serialized_authn": Object {"kind": Object {"Authenticated": Array [Object {"actor": Object {"SiloUser": Object {"silo_id": String("001de000-5110-4000-8000-000000000000"), "silo_user_id": String("001de000-05e4-4000-8000-000000004007")}}}, Object {"mapped_fleet_roles": Object {}}]}}}, "which": Number(1)}, "network_interface_params2": Object {"instance_id": String("d7bd2deb-ee27-4661-9e9a-524d734cd6ec"), "new_id": String("628c4c7e-b430-4a2a-bfd4-c1a8a5a74fa6"), "saga_params": Object {"boundary_switches": Array [], "create_params": Object {"anti_affinity_groups": Array [], "auto_restart_policy": Null, "boot_disk": Null, "description": String("instance \\"utilization-test-instance\\""), "disks": Array [], "external_ips": Array [], "hostname": String("the-host"), "memory": Number(1073741824), "name": String("utilization-test-instance"), "ncpus": Number(4), "network_interfaces": Object {"type": String("default")}, "ssh_public_keys": Array [], "start": Bool(true), "user_data": String("I2Nsb3VkLWNvbmZpZwpzeXN0ZW1faW5mbzoKICBkZWZhdWx0X3VzZXI6CiAgICBuYW1lOiBveGlkZQ==")}, "project_id": String("473b39f9-8669-45b3-b3d7-793f7e10fc31"), "serialized_authn": Object {"kind": Object {"Authenticated": Array [Object {"actor": Object {"SiloUser": Object {"silo_id": String("001de000-5110-4000-8000-000000000000"), "silo_user_id": String("001de000-05e4-4000-8000-000000004007")}}}, Object {"mapped_fleet_roles": Object {}}]}}}, "which": Number(2)}, "network_interface_params3": Object {"instance_id": String("d7bd2deb-ee27-4661-9e9a-524d734cd6ec"), "new_id": String("9b0302b2-2547-4705-b879-b2d896fa654b"), "saga_params": Object {"boundary_switches": Array [], "create_params": Object {"anti_affinity_groups": Array [], "auto_restart_policy": Null, "boot_disk": Null, "description": String("instance \\"utilization-test-instance\\""), "disks": Array [], "external_ips": Array [], "hostname": String("the-host"), "memory": Number(1073741824), "name": String("utilization-test-instance"), "ncpus": Number(4), "network_interfaces": Object {"type": String("default")}, "ssh_public_keys": Array [], "start": Bool(true), "user_data": String("I2Nsb3VkLWNvbmZpZwpzeXN0ZW1faW5mbzoKICBkZWZhdWx0X3VzZXI6CiAgICBuYW1lOiBveGlkZQ==")}, "project_id": String("473b39f9-8669-45b3-b3d7-793f7e10fc31"), "serialized_authn": Object {"kind": Object {"Authenticated": Array [Object {"actor": Object {"SiloUser": Object {"silo_id": String("001de000-5110-4000-8000-000000000000"), "silo_user_id": String("001de000-05e4-4000-8000-000000004007")}}}, Object {"mapped_fleet_roles": Object {}}]}}}, "which": Number(3)}, "network_interface_params4": Object {"instance_id": String("d7bd2deb-ee27-4661-9e9a-524d734cd6ec"), "new_id": String("7390678b-5d44-43e6-b46d-3c25a5eebd71"), "saga_params": Object {"boundary_switches": Array [], "create_params": Object {"anti_affinity_groups": Array [], "auto_restart_policy": Null, "boot_disk": Null, "description": String("instance \\"utilization-test-instance\\""), "disks": Array [], "external_ips": Array [], "hostname": String("the-host"), "memory": Number(1073741824), "name": String("utilization-test-instance"), "ncpus": Number(4), "network_interfaces": Object {"type": String("default")}, "ssh_public_keys": Array [], "start": Bool(true), "user_data": String("I2Nsb3VkLWNvbmZpZwpzeXN0ZW1faW5mbzoKICBkZWZhdWx0X3VzZXI6CiAgICBuYW1lOiBveGlkZQ==")}, "project_id": String("473b39f9-8669-45b3-b3d7-793f7e10fc31"), "serialized_authn": Object {"kind": Object {"Authenticated": Array [Object {"actor": Object {"SiloUser": Object {"silo_id": String("001de000-5110-4000-8000-000000000000"), "silo_user_id": String("001de000-05e4-4000-8000-000000004007")}}}, Object {"mapped_fleet_roles": Object {}}]}}}, "which": Number(4)}, "network_interface_params5": Object {"instance_id": String("d7bd2deb-ee27-4661-9e9a-524d734cd6ec"), "new_id": String("2a09f6d6-3de1-4bea-b89c-6fc00eaf504e"), "saga_params": Object {"boundary_switches": Array [], "create_params": Object {"anti_affinity_groups": Array [], "auto_restart_policy": Null, "boot_disk": Null, "description": String("instance \\"utilization-test-instance\\""), "disks": Array [], "external_ips": Array [], "hostname": String("the-host"), "memory": Number(1073741824), "name": String("utilization-test-instance"), "ncpus": Number(4), "network_interfaces": Object {"type": String("default")}, "ssh_public_keys": Array [], "start": Bool(true), "user_data": String("I2Nsb3VkLWNvbmZpZwpzeXN0ZW1faW5mbzoKICBkZWZhdWx0X3VzZXI6CiAgICBuYW1lOiBveGlkZQ==")}, "project_id": String("473b39f9-8669-45b3-b3d7-793f7e10fc31"), "serialized_authn": Object {"kind": Object {"Authenticated": Array [Object {"actor": Object {"SiloUser": Object {"silo_id": String("001de000-5110-4000-8000-000000000000"), "silo_user_id": String("001de000-05e4-4000-8000-000000004007")}}}, Object {"mapped_fleet_roles": Object {}}]}}}, "which": Number(5)}, "network_interface_params6": Object {"instance_id": String("d7bd2deb-ee27-4661-9e9a-524d734cd6ec"), "new_id": String("86338488-d577-4ce7-a835-4c7491bc0d88"), "saga_params": Object {"boundary_switches": Array [], "create_params": Object {"anti_affinity_groups": Array [], "auto_restart_policy": Null, "boot_disk": Null, "description": String("instance \\"utilization-test-instance\\""), "disks": Array [], "external_ips": Array [], "hostname": String("the-host"), "memory": Number(1073741824), "name": String("utilization-test-instance"), "ncpus": Number(4), "network_interfaces": Object {"type": String("default")}, "ssh_public_keys": Array [], "start": Bool(true), "user_data": String("I2Nsb3VkLWNvbmZpZwpzeXN0ZW1faW5mbzoKICBkZWZhdWx0X3VzZXI6CiAgICBuYW1lOiBveGlkZQ==")}, "project_id": String("473b39f9-8669-45b3-b3d7-793f7e10fc31"), "serialized_authn": Object {"kind": Object {"Authenticated": Array [Object {"actor": Object {"SiloUser": Object {"silo_id": String("001de000-5110-4000-8000-000000000000"), "silo_user_id": String("001de000-05e4-4000-8000-000000004007")}}}, Object {"mapped_fleet_roles": Object {}}]}}}, "which": Number(6)}, "network_interface_params7": Object {"instance_id": String("d7bd2deb-ee27-4661-9e9a-524d734cd6ec"), "new_id": String("d7156580-5c17-44d4-bbc0-c78d1b1133df"), "saga_params": Object {"boundary_switches": Array [], "create_params": Object {"anti_affinity_groups": Array [], "auto_restart_policy": Null, "boot_disk": Null, "description": String("instance \\"utilization-test-instance\\""), "disks": Array [], "external_ips": Array [], "hostname": String("the-host"), "memory": Number(1073741824), "name": String("utilization-test-instance"), "ncpus": Number(4), "network_interfaces": Object {"type": String("default")}, "ssh_public_keys": Array [], "start": Bool(true), "user_data": String("I2Nsb3VkLWNvbmZpZwpzeXN0ZW1faW5mbzoKICBkZWZhdWx0X3VzZXI6CiAgICBuYW1lOiBveGlkZQ==")}, "project_id": String("473b39f9-8669-45b3-b3d7-793f7e10fc31"), "serialized_authn": Object {"kind": Object {"Authenticated": Array [Object {"actor": Object {"SiloUser": Object {"silo_id": String("001de000-5110-4000-8000-000000000000"), "silo_user_id": String("001de000-05e4-4000-8000-000000004007")}}}, Object {"mapped_fleet_roles": Object {}}]}}}, "which": Number(7)}, "output": Null, "set_boot_disk": Null, "snat_ip": Null, "snat_ip_id": String("703ac9cc-cf3f-44f5-9e01-0a385ec2b26a"), "stopped_instance": Null} }) }
50082025-04-30T06:43:58.077ZTRACtest_utilization (dropshot_external): authorize begin action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true local_addr = 127.0.0.1:46348 method = POST remote_addr = 127.0.0.1:41283 req_id = bf272e5e-cbe5-41d9-b505-34877ffe12d2 resource = Database uri = /v1/instances?project=utilization-test-project
50092025-04-30T06:43:58.077ZDEBGtest_utilization (dropshot_external): roles actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true local_addr = 127.0.0.1:46348 method = POST remote_addr = 127.0.0.1:41283 req_id = bf272e5e-cbe5-41d9-b505-34877ffe12d2 roles = RoleSet { roles: {} } uri = /v1/instances?project=utilization-test-project
50102025-04-30T06:43:58.077ZDEBGtest_utilization (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true local_addr = 127.0.0.1:46348 method = POST remote_addr = 127.0.0.1:41283 req_id = bf272e5e-cbe5-41d9-b505-34877ffe12d2 resource = Database result = Ok(()) uri = /v1/instances?project=utilization-test-project
50112025-04-30T06:43:58.080ZTRACtest_utilization (dropshot_external): authorize begin action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true local_addr = 127.0.0.1:46348 method = POST remote_addr = 127.0.0.1:41283 req_id = bf272e5e-cbe5-41d9-b505-34877ffe12d2 resource = Database uri = /v1/instances?project=utilization-test-project
50122025-04-30T06:43:58.080ZDEBGtest_utilization (dropshot_external): roles actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true local_addr = 127.0.0.1:46348 method = POST remote_addr = 127.0.0.1:41283 req_id = bf272e5e-cbe5-41d9-b505-34877ffe12d2 roles = RoleSet { roles: {} } uri = /v1/instances?project=utilization-test-project
50132025-04-30T06:43:58.081ZDEBGtest_utilization (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true local_addr = 127.0.0.1:46348 method = POST remote_addr = 127.0.0.1:41283 req_id = bf272e5e-cbe5-41d9-b505-34877ffe12d2 resource = Database result = Ok(()) uri = /v1/instances?project=utilization-test-project
50142025-04-30T06:43:58.084ZTRACtest_utilization (dropshot_external): authorize begin action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true local_addr = 127.0.0.1:46348 method = POST remote_addr = 127.0.0.1:41283 req_id = bf272e5e-cbe5-41d9-b505-34877ffe12d2 resource = Database uri = /v1/instances?project=utilization-test-project
50152025-04-30T06:43:58.084ZDEBGtest_utilization (dropshot_external): roles actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true local_addr = 127.0.0.1:46348 method = POST remote_addr = 127.0.0.1:41283 req_id = bf272e5e-cbe5-41d9-b505-34877ffe12d2 roles = RoleSet { roles: {} } uri = /v1/instances?project=utilization-test-project
50162025-04-30T06:43:58.085ZDEBGtest_utilization (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true local_addr = 127.0.0.1:46348 method = POST remote_addr = 127.0.0.1:41283 req_id = bf272e5e-cbe5-41d9-b505-34877ffe12d2 resource = Database result = Ok(()) uri = /v1/instances?project=utilization-test-project
50172025-04-30T06:43:58.088ZTRACtest_utilization (dropshot_external): authorize begin action = Modify actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true local_addr = 127.0.0.1:46348 method = POST remote_addr = 127.0.0.1:41283 req_id = bf272e5e-cbe5-41d9-b505-34877ffe12d2 resource = Instance { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000000, lookup_type: ById(001de000-5110-4000-8000-000000000000) }, key: 473b39f9-8669-45b3-b3d7-793f7e10fc31, lookup_type: ById(473b39f9-8669-45b3-b3d7-793f7e10fc31) }, key: d7bd2deb-ee27-4661-9e9a-524d734cd6ec, lookup_type: ById(d7bd2deb-ee27-4661-9e9a-524d734cd6ec) } uri = /v1/instances?project=utilization-test-project
50182025-04-30T06:43:58.088ZTRACtest_utilization (dropshot_external): loading roles actor = Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. } actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true local_addr = 127.0.0.1:46348 method = POST remote_addr = 127.0.0.1:41283 req_id = bf272e5e-cbe5-41d9-b505-34877ffe12d2 resource_id = 473b39f9-8669-45b3-b3d7-793f7e10fc31 resource_type = Project uri = /v1/instances?project=utilization-test-project
50192025-04-30T06:43:58.088ZTRACtest_utilization (dropshot_external): authorize begin action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true local_addr = 127.0.0.1:46348 method = POST remote_addr = 127.0.0.1:41283 req_id = bf272e5e-cbe5-41d9-b505-34877ffe12d2 resource = Database uri = /v1/instances?project=utilization-test-project
50202025-04-30T06:43:58.088ZDEBGtest_utilization (dropshot_external): roles actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true local_addr = 127.0.0.1:46348 method = POST remote_addr = 127.0.0.1:41283 req_id = bf272e5e-cbe5-41d9-b505-34877ffe12d2 roles = RoleSet { roles: {} } uri = /v1/instances?project=utilization-test-project
50212025-04-30T06:43:58.088ZDEBGtest_utilization (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true local_addr = 127.0.0.1:46348 method = POST remote_addr = 127.0.0.1:41283 req_id = bf272e5e-cbe5-41d9-b505-34877ffe12d2 resource = Database result = Ok(()) uri = /v1/instances?project=utilization-test-project
50222025-04-30T06:43:58.092ZTRACtest_utilization (dropshot_external): loading roles actor = Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. } actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true local_addr = 127.0.0.1:46348 method = POST remote_addr = 127.0.0.1:41283 req_id = bf272e5e-cbe5-41d9-b505-34877ffe12d2 resource_id = 001de000-5110-4000-8000-000000000000 resource_type = Silo uri = /v1/instances?project=utilization-test-project
50232025-04-30T06:43:58.092ZTRACtest_utilization (dropshot_external): authorize begin action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true local_addr = 127.0.0.1:46348 method = POST remote_addr = 127.0.0.1:41283 req_id = bf272e5e-cbe5-41d9-b505-34877ffe12d2 resource = Database uri = /v1/instances?project=utilization-test-project
50242025-04-30T06:43:58.092ZDEBGtest_utilization (dropshot_external): roles actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true local_addr = 127.0.0.1:46348 method = POST remote_addr = 127.0.0.1:41283 req_id = bf272e5e-cbe5-41d9-b505-34877ffe12d2 roles = RoleSet { roles: {} } uri = /v1/instances?project=utilization-test-project
50252025-04-30T06:43:58.093ZDEBGtest_utilization (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true local_addr = 127.0.0.1:46348 method = POST remote_addr = 127.0.0.1:41283 req_id = bf272e5e-cbe5-41d9-b505-34877ffe12d2 resource = Database result = Ok(()) uri = /v1/instances?project=utilization-test-project
50262025-04-30T06:43:58.096ZTRACtest_utilization (dropshot_external): loading roles actor = Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. } actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true local_addr = 127.0.0.1:46348 method = POST remote_addr = 127.0.0.1:41283 req_id = bf272e5e-cbe5-41d9-b505-34877ffe12d2 resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet uri = /v1/instances?project=utilization-test-project
50272025-04-30T06:43:58.096ZTRACtest_utilization (dropshot_external): authorize begin action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true local_addr = 127.0.0.1:46348 method = POST remote_addr = 127.0.0.1:41283 req_id = bf272e5e-cbe5-41d9-b505-34877ffe12d2 resource = Database uri = /v1/instances?project=utilization-test-project
50282025-04-30T06:43:58.096ZDEBGtest_utilization (dropshot_external): roles actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true local_addr = 127.0.0.1:46348 method = POST remote_addr = 127.0.0.1:41283 req_id = bf272e5e-cbe5-41d9-b505-34877ffe12d2 roles = RoleSet { roles: {} } uri = /v1/instances?project=utilization-test-project
50292025-04-30T06:43:58.097ZDEBGtest_utilization (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true local_addr = 127.0.0.1:46348 method = POST remote_addr = 127.0.0.1:41283 req_id = bf272e5e-cbe5-41d9-b505-34877ffe12d2 resource = Database result = Ok(()) uri = /v1/instances?project=utilization-test-project
50302025-04-30T06:43:58.101ZDEBGtest_utilization (dropshot_external): roles actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true local_addr = 127.0.0.1:46348 method = POST remote_addr = 127.0.0.1:41283 req_id = bf272e5e-cbe5-41d9-b505-34877ffe12d2 roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin"), (Silo, 001de000-5110-4000-8000-000000000000, "admin")} } uri = /v1/instances?project=utilization-test-project
50312025-04-30T06:43:58.108ZDEBGtest_utilization (dropshot_external): authorize result action = Modify actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true local_addr = 127.0.0.1:46348 method = POST remote_addr = 127.0.0.1:41283 req_id = bf272e5e-cbe5-41d9-b505-34877ffe12d2 resource = Instance { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000000, lookup_type: ById(001de000-5110-4000-8000-000000000000) }, key: 473b39f9-8669-45b3-b3d7-793f7e10fc31, lookup_type: ById(473b39f9-8669-45b3-b3d7-793f7e10fc31) }, key: d7bd2deb-ee27-4661-9e9a-524d734cd6ec, lookup_type: ById(d7bd2deb-ee27-4661-9e9a-524d734cd6ec) } result = Ok(()) uri = /v1/instances?project=utilization-test-project
50322025-04-30T06:43:58.108ZTRACtest_utilization (dropshot_external): authorize begin action = Read actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true local_addr = 127.0.0.1:46348 method = POST remote_addr = 127.0.0.1:41283 req_id = bf272e5e-cbe5-41d9-b505-34877ffe12d2 resource = Instance { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000000, lookup_type: ById(001de000-5110-4000-8000-000000000000) }, key: 473b39f9-8669-45b3-b3d7-793f7e10fc31, lookup_type: ById(473b39f9-8669-45b3-b3d7-793f7e10fc31) }, key: d7bd2deb-ee27-4661-9e9a-524d734cd6ec, lookup_type: ById(d7bd2deb-ee27-4661-9e9a-524d734cd6ec) } uri = /v1/instances?project=utilization-test-project
50332025-04-30T06:43:58.108ZTRACtest_utilization (dropshot_external): loading roles actor = Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. } actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true local_addr = 127.0.0.1:46348 method = POST remote_addr = 127.0.0.1:41283 req_id = bf272e5e-cbe5-41d9-b505-34877ffe12d2 resource_id = 473b39f9-8669-45b3-b3d7-793f7e10fc31 resource_type = Project uri = /v1/instances?project=utilization-test-project
50342025-04-30T06:43:58.108ZTRACtest_utilization (dropshot_external): authorize begin action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true local_addr = 127.0.0.1:46348 method = POST remote_addr = 127.0.0.1:41283 req_id = bf272e5e-cbe5-41d9-b505-34877ffe12d2 resource = Database uri = /v1/instances?project=utilization-test-project
50352025-04-30T06:43:58.108ZDEBGtest_utilization (dropshot_external): roles actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true local_addr = 127.0.0.1:46348 method = POST remote_addr = 127.0.0.1:41283 req_id = bf272e5e-cbe5-41d9-b505-34877ffe12d2 roles = RoleSet { roles: {} } uri = /v1/instances?project=utilization-test-project
50362025-04-30T06:43:58.108ZDEBGtest_utilization (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true local_addr = 127.0.0.1:46348 method = POST remote_addr = 127.0.0.1:41283 req_id = bf272e5e-cbe5-41d9-b505-34877ffe12d2 resource = Database result = Ok(()) uri = /v1/instances?project=utilization-test-project
50372025-04-30T06:43:58.122ZTRACtest_utilization (dropshot_external): loading roles actor = Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. } actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true local_addr = 127.0.0.1:46348 method = POST remote_addr = 127.0.0.1:41283 req_id = bf272e5e-cbe5-41d9-b505-34877ffe12d2 resource_id = 001de000-5110-4000-8000-000000000000 resource_type = Silo uri = /v1/instances?project=utilization-test-project
50382025-04-30T06:43:58.122ZTRACtest_utilization (dropshot_external): authorize begin action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true local_addr = 127.0.0.1:46348 method = POST remote_addr = 127.0.0.1:41283 req_id = bf272e5e-cbe5-41d9-b505-34877ffe12d2 resource = Database uri = /v1/instances?project=utilization-test-project
50392025-04-30T06:43:58.122ZDEBGtest_utilization (dropshot_external): roles actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true local_addr = 127.0.0.1:46348 method = POST remote_addr = 127.0.0.1:41283 req_id = bf272e5e-cbe5-41d9-b505-34877ffe12d2 roles = RoleSet { roles: {} } uri = /v1/instances?project=utilization-test-project
50402025-04-30T06:43:58.125ZDEBGtest_utilization (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true local_addr = 127.0.0.1:46348 method = POST remote_addr = 127.0.0.1:41283 req_id = bf272e5e-cbe5-41d9-b505-34877ffe12d2 resource = Database result = Ok(()) uri = /v1/instances?project=utilization-test-project
50412025-04-30T06:43:58.129ZTRACtest_utilization (dropshot_external): loading roles actor = Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. } actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true local_addr = 127.0.0.1:46348 method = POST remote_addr = 127.0.0.1:41283 req_id = bf272e5e-cbe5-41d9-b505-34877ffe12d2 resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet uri = /v1/instances?project=utilization-test-project
50422025-04-30T06:43:58.130ZTRACtest_utilization (dropshot_external): authorize begin action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true local_addr = 127.0.0.1:46348 method = POST remote_addr = 127.0.0.1:41283 req_id = bf272e5e-cbe5-41d9-b505-34877ffe12d2 resource = Database uri = /v1/instances?project=utilization-test-project
50432025-04-30T06:43:58.130ZDEBGtest_utilization (dropshot_external): roles actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true local_addr = 127.0.0.1:46348 method = POST remote_addr = 127.0.0.1:41283 req_id = bf272e5e-cbe5-41d9-b505-34877ffe12d2 roles = RoleSet { roles: {} } uri = /v1/instances?project=utilization-test-project
50442025-04-30T06:43:58.131ZDEBGtest_utilization (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true local_addr = 127.0.0.1:46348 method = POST remote_addr = 127.0.0.1:41283 req_id = bf272e5e-cbe5-41d9-b505-34877ffe12d2 resource = Database result = Ok(()) uri = /v1/instances?project=utilization-test-project
50452025-04-30T06:43:58.135ZDEBGtest_utilization (dropshot_external): roles actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true local_addr = 127.0.0.1:46348 method = POST remote_addr = 127.0.0.1:41283 req_id = bf272e5e-cbe5-41d9-b505-34877ffe12d2 roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin"), (Silo, 001de000-5110-4000-8000-000000000000, "admin")} } uri = /v1/instances?project=utilization-test-project
50462025-04-30T06:43:58.143ZDEBGtest_utilization (dropshot_external): authorize result action = Read actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true local_addr = 127.0.0.1:46348 method = POST remote_addr = 127.0.0.1:41283 req_id = bf272e5e-cbe5-41d9-b505-34877ffe12d2 resource = Instance { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000000, lookup_type: ById(001de000-5110-4000-8000-000000000000) }, key: 473b39f9-8669-45b3-b3d7-793f7e10fc31, lookup_type: ById(473b39f9-8669-45b3-b3d7-793f7e10fc31) }, key: d7bd2deb-ee27-4661-9e9a-524d734cd6ec, lookup_type: ById(d7bd2deb-ee27-4661-9e9a-524d734cd6ec) } result = Ok(()) uri = /v1/instances?project=utilization-test-project
50472025-04-30T06:43:58.143ZTRACtest_utilization (dropshot_external): authorize begin action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true local_addr = 127.0.0.1:46348 method = POST remote_addr = 127.0.0.1:41283 req_id = bf272e5e-cbe5-41d9-b505-34877ffe12d2 resource = Database uri = /v1/instances?project=utilization-test-project
50482025-04-30T06:43:58.143ZDEBGtest_utilization (dropshot_external): roles actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true local_addr = 127.0.0.1:46348 method = POST remote_addr = 127.0.0.1:41283 req_id = bf272e5e-cbe5-41d9-b505-34877ffe12d2 roles = RoleSet { roles: {} } uri = /v1/instances?project=utilization-test-project
50492025-04-30T06:43:58.143ZDEBGtest_utilization (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true local_addr = 127.0.0.1:46348 method = POST remote_addr = 127.0.0.1:41283 req_id = bf272e5e-cbe5-41d9-b505-34877ffe12d2 resource = Database result = Ok(()) uri = /v1/instances?project=utilization-test-project
50502025-04-30T06:43:58.149ZINFOtest_utilization (SagaExecutor): preparing saga saga_id = bc1ac890-e9e2-44d9-931e-3abe24cf75f5 saga_name = instance-start
50512025-04-30T06:43:58.150ZINFOtest_utilization (SEC): saga create dag = {"end_node":11,"graph":{"edge_property":"directed","edges":[[0,1,null],[1,2,null],[2,3,null],[3,4,null],[4,5,null],[5,6,null],[6,7,null],[7,8,null],[8,9,null],[10,0,null],[9,11,null]],"node_holes":[],"nodes":[{"Action":{"action_name":"instance_start.generate_propolis_id","label":"GeneratePropolisId","name":"propolis_id"}},{"Action":{"action_name":"instance_start.alloc_server","label":"AllocServer","name":"sled_id"}},{"Action":{"action_name":"instance_start.alloc_propolis_ip","label":"AllocPropolisIp","name":"propolis_ip"}},{"Action":{"action_name":"instance_start.create_vmm_record","label":"CreateVmmRecord","name":"vmm_record"}},{"Action":{"action_name":"instance_start.mark_as_starting","label":"MarkAsStarting","name":"started_record"}},{"Action":{"action_name":"instance_start.dpd_ensure","label":"DpdEnsure","name":"dpd_ensure"}},{"Action":{"action_name":"instance_start.v2p_ensure","label":"V2PEnsure","name":"v2p_ensure"}},{"Action":{"action_name":"instance_start.ensure_registered","label":"EnsureRegistered","name":"ensure_registered"}},{"Action":{"action_name":"instance_start.add_virtual_resources","label":"AddVirtualResources","name":"virtual_resources"}},{"Action":{"action_name":"instance_start.ensure_running","label":"EnsureRunning","name":"ensure_running"}},{"Start":{"params":{"db_instance":{"auto_restart":{"cooldown":null,"policy":null},"boot_disk_id":null,"hostname":"the-host","identity":{"description":"instance \\"utilization-test-instance\\"","id":"d7bd2deb-ee27-4661-9e9a-524d734cd6ec","name":"utilization-test-instance","time_created":"2025-04-30T06:43:54.249576Z","time_deleted":null,"time_modified":"2025-04-30T06:43:54.249576Z"},"memory":1073741824,"ncpus":4,"project_id":"473b39f9-8669-45b3-b3d7-793f7e10fc31","runtime_state":{"dst_propolis_id":null,"gen":2,"migration_id":null,"nexus_state":"NoVmm","propolis_id":null,"time_last_auto_restarted":null,"time_updated":"2025-04-30T06:43:54.249576Z"},"updater_gen":1,"updater_id":null,"user_data":[35,99,108,111,117,100,45,99,111,110,102,105,103,10,115,121,115,116,101,109,95,105,110,102,111,58,10,32,32,100,101,102,97,117,108,116,95,117,115,101,114,58,10,32,32,32,32,110,97,109,101,58,32,111,120,105,100,101]},"reason":"AutoStart","serialized_authn":{"kind":{"Authenticated":[{"actor":{"SiloUser":{"silo_id":"001de000-5110-4000-8000-000000000000","silo_user_id":"001de000-05e4-4000-8000-000000004007"}}},{"mapped_fleet_roles":{}}]}}}}},"End"]},"saga_name":"instance-start","start_node":10} saga_id = bc1ac890-e9e2-44d9-931e-3abe24cf75f5 saga_name = instance-start sec_id = ac55d8d9-a298-4534-898b-beee1c9e4e52
50522025-04-30T06:43:58.150ZINFOtest_utilization (SecStore): creating saga saga_id = bc1ac890-e9e2-44d9-931e-3abe24cf75f5 saga_name = instance-start
50532025-04-30T06:43:58.156ZINFOtest_utilization (SagaExecutor): starting saga saga_id = bc1ac890-e9e2-44d9-931e-3abe24cf75f5 saga_name = instance-start
50542025-04-30T06:43:58.156ZINFOtest_utilization (SEC): saga start saga_id = bc1ac890-e9e2-44d9-931e-3abe24cf75f5 saga_name = instance-start sec_id = ac55d8d9-a298-4534-898b-beee1c9e4e52
50552025-04-30T06:43:58.157ZDEBGtest_utilization (SEC): saga log event new_state = N010 started sec_id = ac55d8d9-a298-4534-898b-beee1c9e4e52
50562025-04-30T06:43:58.157ZDEBGtest_utilization (SecStore): recording saga event event_type = Started node_id = 10 saga_id = bc1ac890-e9e2-44d9-931e-3abe24cf75f5
50572025-04-30T06:43:58.164ZDEBGtest_utilization (SEC): saga log event new_state = N010 succeeded sec_id = ac55d8d9-a298-4534-898b-beee1c9e4e52
50582025-04-30T06:43:58.164ZDEBGtest_utilization (SecStore): recording saga event event_type = Succeeded(Null) node_id = 10 saga_id = bc1ac890-e9e2-44d9-931e-3abe24cf75f5
50592025-04-30T06:43:58.169ZDEBGtest_utilization (SEC): saga log event new_state = N000 started sec_id = ac55d8d9-a298-4534-898b-beee1c9e4e52
50602025-04-30T06:43:58.169ZDEBGtest_utilization (SecStore): recording saga event event_type = Started node_id = 0 saga_id = bc1ac890-e9e2-44d9-931e-3abe24cf75f5
50612025-04-30T06:43:58.176ZDEBGtest_utilization (SEC): saga log event new_state = N000 succeeded sec_id = ac55d8d9-a298-4534-898b-beee1c9e4e52
50622025-04-30T06:43:58.176ZDEBGtest_utilization (SecStore): recording saga event event_type = Succeeded(String("9e8dbed7-faa3-4634-886b-3a005dd8e99b")) node_id = 0 saga_id = bc1ac890-e9e2-44d9-931e-3abe24cf75f5
50632025-04-30T06:43:58.191ZDEBGtest_utilization (SEC): saga log event new_state = N001 started sec_id = ac55d8d9-a298-4534-898b-beee1c9e4e52
50642025-04-30T06:43:58.191ZDEBGtest_utilization (SecStore): recording saga event event_type = Started node_id = 1 saga_id = bc1ac890-e9e2-44d9-931e-3abe24cf75f5
50652025-04-30T06:43:58.206ZTRACtest_utilization (InstanceAllocator): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000004ead, .. }) resource = Database
50662025-04-30T06:43:58.206ZDEBGtest_utilization (InstanceAllocator): roles roles = RoleSet { roles: {} }
50672025-04-30T06:43:58.207ZDEBGtest_utilization (InstanceAllocator): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000004ead, .. }) resource = Database result = Ok(())
50682025-04-30T06:43:58.267ZDEBGtest_utilization (SEC): saga log event new_state = N001 succeeded sec_id = ac55d8d9-a298-4534-898b-beee1c9e4e52
50692025-04-30T06:43:58.267ZDEBGtest_utilization (SecStore): recording saga event event_type = Succeeded(String("b6d65341-167c-41df-9b5c-41cded99c229")) node_id = 1 saga_id = bc1ac890-e9e2-44d9-931e-3abe24cf75f5
50702025-04-30T06:43:58.274ZDEBGtest_utilization (SEC): saga log event new_state = N002 started sec_id = ac55d8d9-a298-4534-898b-beee1c9e4e52
50712025-04-30T06:43:58.274ZDEBGtest_utilization (SecStore): recording saga event event_type = Started node_id = 2 saga_id = bc1ac890-e9e2-44d9-931e-3abe24cf75f5
50722025-04-30T06:43:58.279ZTRACtest_utilization (SagaExecutor): authorize begin action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true resource = Database saga_id = bc1ac890-e9e2-44d9-931e-3abe24cf75f5 saga_name = instance-start saga_node = AllocPropolisIp
50732025-04-30T06:43:58.292ZDEBGtest_utilization (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true roles = RoleSet { roles: {} } saga_id = bc1ac890-e9e2-44d9-931e-3abe24cf75f5 saga_name = instance-start saga_node = AllocPropolisIp
50742025-04-30T06:43:58.293ZDEBGtest_utilization (SagaExecutor): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true resource = Database result = Ok(()) saga_id = bc1ac890-e9e2-44d9-931e-3abe24cf75f5 saga_name = instance-start saga_node = AllocPropolisIp
50752025-04-30T06:43:58.301ZDEBGtest_utilization (SEC): saga log event new_state = N002 succeeded sec_id = ac55d8d9-a298-4534-898b-beee1c9e4e52
50762025-04-30T06:43:58.301ZDEBGtest_utilization (SecStore): recording saga event event_type = Succeeded(String("::1:0")) node_id = 2 saga_id = bc1ac890-e9e2-44d9-931e-3abe24cf75f5
50772025-04-30T06:43:58.335ZDEBGtest_utilization (SEC): saga log event new_state = N003 started sec_id = ac55d8d9-a298-4534-898b-beee1c9e4e52
50782025-04-30T06:43:58.335ZDEBGtest_utilization (SecStore): recording saga event event_type = Started node_id = 3 saga_id = bc1ac890-e9e2-44d9-931e-3abe24cf75f5
50792025-04-30T06:43:58.339ZTRACtest_utilization (SagaExecutor): authorize begin action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true resource = Database saga_id = bc1ac890-e9e2-44d9-931e-3abe24cf75f5 saga_name = instance-start saga_node = CreateVmmRecord
50802025-04-30T06:43:58.339ZDEBGtest_utilization (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true roles = RoleSet { roles: {} } saga_id = bc1ac890-e9e2-44d9-931e-3abe24cf75f5 saga_name = instance-start saga_node = CreateVmmRecord
50812025-04-30T06:43:58.340ZDEBGtest_utilization (SagaExecutor): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true resource = Database result = Ok(()) saga_id = bc1ac890-e9e2-44d9-931e-3abe24cf75f5 saga_name = instance-start saga_node = CreateVmmRecord
50822025-04-30T06:43:58.357ZDEBGtest_utilization (results-sink): inserting 1 samples into database collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
50832025-04-30T06:43:58.357ZDEBGtest_utilization (results-sink-cluster): inserting 1 samples into database collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
50842025-04-30T06:43:58.357ZDEBGtest_utilization (clickhouse-client): unrolling 1 total samples collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 4d1485f0-110e-4161-9a33-c4610d68be5d
50852025-04-30T06:43:58.358ZTRACtest_utilization (clickhouse-client): inserting data collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 4d1485f0-110e-4161-9a33-c4610d68be5d n_columns = 4 n_rows = 2 sql = INSERT INTO oximeter.fields_string FORMAT Native
50862025-04-30T06:43:58.358ZDEBGtest_utilization (SEC): saga log event new_state = N003 succeeded sec_id = ac55d8d9-a298-4534-898b-beee1c9e4e52
50872025-04-30T06:43:58.358ZDEBGtest_utilization (SecStore): recording saga event event_type = Succeeded(Object {"id": String("9e8dbed7-faa3-4634-886b-3a005dd8e99b"), "instance_id": String("d7bd2deb-ee27-4661-9e9a-524d734cd6ec"), "propolis_ip": String("::1:0/128"), "propolis_port": Number(12400), "runtime": Object {"gen": Number(1), "state": String("Creating"), "time_state_updated": String("2025-04-30T06:43:58.339117Z")}, "sled_id": String("b6d65341-167c-41df-9b5c-41cded99c229"), "time_created": String("2025-04-30T06:43:58.339117Z"), "time_deleted": Null}) node_id = 3 saga_id = bc1ac890-e9e2-44d9-931e-3abe24cf75f5
50882025-04-30T06:43:58.374ZDEBGtest_utilization (clickhouse-client): inserted rows into table collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 4d1485f0-110e-4161-9a33-c4610d68be5d n_rows = 2 table_name = fields_string
50892025-04-30T06:43:58.374ZTRACtest_utilization (clickhouse-client): inserting data collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 4d1485f0-110e-4161-9a33-c4610d68be5d n_columns = 4 n_rows = 1 sql = INSERT INTO oximeter.measurements_i64 FORMAT Native
50902025-04-30T06:43:58.390ZDEBGtest_utilization (SEC): saga log event new_state = N004 started sec_id = ac55d8d9-a298-4534-898b-beee1c9e4e52
50912025-04-30T06:43:58.390ZDEBGtest_utilization (SecStore): recording saga event event_type = Started node_id = 4 saga_id = bc1ac890-e9e2-44d9-931e-3abe24cf75f5
50922025-04-30T06:43:58.396ZINFOtest_utilization (SagaExecutor): moving instance to Starting state via saga instance_id = d7bd2deb-ee27-4661-9e9a-524d734cd6ec propolis_id = 9e8dbed7-faa3-4634-886b-3a005dd8e99b saga_id = bc1ac890-e9e2-44d9-931e-3abe24cf75f5 saga_name = instance-start start_reason = AutoStart
50932025-04-30T06:43:58.396ZTRACtest_utilization (SagaExecutor): authorize begin action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true resource = Database saga_id = bc1ac890-e9e2-44d9-931e-3abe24cf75f5 saga_name = instance-start saga_node = MarkAsStarting
50942025-04-30T06:43:58.396ZDEBGtest_utilization (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true roles = RoleSet { roles: {} } saga_id = bc1ac890-e9e2-44d9-931e-3abe24cf75f5 saga_name = instance-start saga_node = MarkAsStarting
50952025-04-30T06:43:58.397ZDEBGtest_utilization (SagaExecutor): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true resource = Database result = Ok(()) saga_id = bc1ac890-e9e2-44d9-931e-3abe24cf75f5 saga_name = instance-start saga_node = MarkAsStarting
50962025-04-30T06:43:58.398ZDEBGtest_utilization (clickhouse-client): inserted rows into table collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 4d1485f0-110e-4161-9a33-c4610d68be5d n_rows = 1 table_name = measurements_i64
50972025-04-30T06:43:58.398ZTRACtest_utilization (results-sink): successfully inserted samples collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
50982025-04-30T06:43:58.404ZTRACtest_utilization (SagaExecutor): authorize begin action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true resource = Database saga_id = bc1ac890-e9e2-44d9-931e-3abe24cf75f5 saga_name = instance-start saga_node = MarkAsStarting
50992025-04-30T06:43:58.404ZDEBGtest_utilization (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true roles = RoleSet { roles: {} } saga_id = bc1ac890-e9e2-44d9-931e-3abe24cf75f5 saga_name = instance-start saga_node = MarkAsStarting
51002025-04-30T06:43:58.404ZDEBGtest_utilization (SagaExecutor): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true resource = Database result = Ok(()) saga_id = bc1ac890-e9e2-44d9-931e-3abe24cf75f5 saga_name = instance-start saga_node = MarkAsStarting
51012025-04-30T06:43:58.437ZTRACtest_utilization (SagaExecutor): authorize begin action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true resource = Database saga_id = bc1ac890-e9e2-44d9-931e-3abe24cf75f5 saga_name = instance-start saga_node = MarkAsStarting
51022025-04-30T06:43:58.437ZDEBGtest_utilization (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true roles = RoleSet { roles: {} } saga_id = bc1ac890-e9e2-44d9-931e-3abe24cf75f5 saga_name = instance-start saga_node = MarkAsStarting
51032025-04-30T06:43:58.438ZDEBGtest_utilization (SagaExecutor): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true resource = Database result = Ok(()) saga_id = bc1ac890-e9e2-44d9-931e-3abe24cf75f5 saga_name = instance-start saga_node = MarkAsStarting
51042025-04-30T06:43:58.449ZTRACtest_utilization (SagaExecutor): authorize begin action = Modify actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true resource = Instance { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000000, lookup_type: ById(001de000-5110-4000-8000-000000000000) }, key: 473b39f9-8669-45b3-b3d7-793f7e10fc31, lookup_type: ById(473b39f9-8669-45b3-b3d7-793f7e10fc31) }, key: d7bd2deb-ee27-4661-9e9a-524d734cd6ec, lookup_type: ById(d7bd2deb-ee27-4661-9e9a-524d734cd6ec) } saga_id = bc1ac890-e9e2-44d9-931e-3abe24cf75f5 saga_name = instance-start saga_node = MarkAsStarting
51052025-04-30T06:43:58.449ZTRACtest_utilization (SagaExecutor): loading roles actor = Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. } actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true resource_id = 473b39f9-8669-45b3-b3d7-793f7e10fc31 resource_type = Project saga_id = bc1ac890-e9e2-44d9-931e-3abe24cf75f5 saga_name = instance-start saga_node = MarkAsStarting
51062025-04-30T06:43:58.449ZTRACtest_utilization (SagaExecutor): authorize begin action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true resource = Database saga_id = bc1ac890-e9e2-44d9-931e-3abe24cf75f5 saga_name = instance-start saga_node = MarkAsStarting
51072025-04-30T06:43:58.449ZDEBGtest_utilization (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true roles = RoleSet { roles: {} } saga_id = bc1ac890-e9e2-44d9-931e-3abe24cf75f5 saga_name = instance-start saga_node = MarkAsStarting
51082025-04-30T06:43:58.450ZDEBGtest_utilization (SagaExecutor): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true resource = Database result = Ok(()) saga_id = bc1ac890-e9e2-44d9-931e-3abe24cf75f5 saga_name = instance-start saga_node = MarkAsStarting
51092025-04-30T06:43:58.455ZTRACtest_utilization (SagaExecutor): loading roles actor = Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. } actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true resource_id = 001de000-5110-4000-8000-000000000000 resource_type = Silo saga_id = bc1ac890-e9e2-44d9-931e-3abe24cf75f5 saga_name = instance-start saga_node = MarkAsStarting
51102025-04-30T06:43:58.455ZTRACtest_utilization (SagaExecutor): authorize begin action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true resource = Database saga_id = bc1ac890-e9e2-44d9-931e-3abe24cf75f5 saga_name = instance-start saga_node = MarkAsStarting
51112025-04-30T06:43:58.455ZDEBGtest_utilization (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true roles = RoleSet { roles: {} } saga_id = bc1ac890-e9e2-44d9-931e-3abe24cf75f5 saga_name = instance-start saga_node = MarkAsStarting
51122025-04-30T06:43:58.456ZDEBGtest_utilization (SagaExecutor): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true resource = Database result = Ok(()) saga_id = bc1ac890-e9e2-44d9-931e-3abe24cf75f5 saga_name = instance-start saga_node = MarkAsStarting
51132025-04-30T06:43:58.460ZWARNtest_utilization (results-sink-cluster): failed to insert some results into metric DB collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 error = Failed to check out connection to database: No backends found for this service file = oximeter/collector/src/results_sink.rs:92
51142025-04-30T06:43:58.468ZTRACtest_utilization (SagaExecutor): loading roles actor = Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. } actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet saga_id = bc1ac890-e9e2-44d9-931e-3abe24cf75f5 saga_name = instance-start saga_node = MarkAsStarting
51152025-04-30T06:43:58.468ZTRACtest_utilization (SagaExecutor): authorize begin action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true resource = Database saga_id = bc1ac890-e9e2-44d9-931e-3abe24cf75f5 saga_name = instance-start saga_node = MarkAsStarting
51162025-04-30T06:43:58.469ZDEBGtest_utilization (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true roles = RoleSet { roles: {} } saga_id = bc1ac890-e9e2-44d9-931e-3abe24cf75f5 saga_name = instance-start saga_node = MarkAsStarting
51172025-04-30T06:43:58.469ZDEBGtest_utilization (SagaExecutor): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true resource = Database result = Ok(()) saga_id = bc1ac890-e9e2-44d9-931e-3abe24cf75f5 saga_name = instance-start saga_node = MarkAsStarting
51182025-04-30T06:43:58.474ZDEBGtest_utilization (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin"), (Silo, 001de000-5110-4000-8000-000000000000, "admin")} } saga_id = bc1ac890-e9e2-44d9-931e-3abe24cf75f5 saga_name = instance-start saga_node = MarkAsStarting
51192025-04-30T06:43:58.490ZDEBGtest_utilization (SagaExecutor): authorize result action = Modify actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true resource = Instance { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000000, lookup_type: ById(001de000-5110-4000-8000-000000000000) }, key: 473b39f9-8669-45b3-b3d7-793f7e10fc31, lookup_type: ById(473b39f9-8669-45b3-b3d7-793f7e10fc31) }, key: d7bd2deb-ee27-4661-9e9a-524d734cd6ec, lookup_type: ById(d7bd2deb-ee27-4661-9e9a-524d734cd6ec) } result = Ok(()) saga_id = bc1ac890-e9e2-44d9-931e-3abe24cf75f5 saga_name = instance-start saga_node = MarkAsStarting
51202025-04-30T06:43:58.490ZTRACtest_utilization (SagaExecutor): authorize begin action = Read actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true resource = Instance { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000000, lookup_type: ById(001de000-5110-4000-8000-000000000000) }, key: 473b39f9-8669-45b3-b3d7-793f7e10fc31, lookup_type: ById(473b39f9-8669-45b3-b3d7-793f7e10fc31) }, key: d7bd2deb-ee27-4661-9e9a-524d734cd6ec, lookup_type: ById(d7bd2deb-ee27-4661-9e9a-524d734cd6ec) } saga_id = bc1ac890-e9e2-44d9-931e-3abe24cf75f5 saga_name = instance-start saga_node = MarkAsStarting
51212025-04-30T06:43:58.490ZTRACtest_utilization (SagaExecutor): loading roles actor = Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. } actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true resource_id = 473b39f9-8669-45b3-b3d7-793f7e10fc31 resource_type = Project saga_id = bc1ac890-e9e2-44d9-931e-3abe24cf75f5 saga_name = instance-start saga_node = MarkAsStarting
51222025-04-30T06:43:58.490ZTRACtest_utilization (SagaExecutor): authorize begin action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true resource = Database saga_id = bc1ac890-e9e2-44d9-931e-3abe24cf75f5 saga_name = instance-start saga_node = MarkAsStarting
51232025-04-30T06:43:58.490ZDEBGtest_utilization (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true roles = RoleSet { roles: {} } saga_id = bc1ac890-e9e2-44d9-931e-3abe24cf75f5 saga_name = instance-start saga_node = MarkAsStarting
51242025-04-30T06:43:58.490ZDEBGtest_utilization (SagaExecutor): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true resource = Database result = Ok(()) saga_id = bc1ac890-e9e2-44d9-931e-3abe24cf75f5 saga_name = instance-start saga_node = MarkAsStarting
51252025-04-30T06:43:58.494ZTRACtest_utilization (SagaExecutor): loading roles actor = Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. } actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true resource_id = 001de000-5110-4000-8000-000000000000 resource_type = Silo saga_id = bc1ac890-e9e2-44d9-931e-3abe24cf75f5 saga_name = instance-start saga_node = MarkAsStarting
51262025-04-30T06:43:58.494ZTRACtest_utilization (SagaExecutor): authorize begin action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true resource = Database saga_id = bc1ac890-e9e2-44d9-931e-3abe24cf75f5 saga_name = instance-start saga_node = MarkAsStarting
51272025-04-30T06:43:58.494ZDEBGtest_utilization (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true roles = RoleSet { roles: {} } saga_id = bc1ac890-e9e2-44d9-931e-3abe24cf75f5 saga_name = instance-start saga_node = MarkAsStarting
51282025-04-30T06:43:58.495ZDEBGtest_utilization (SagaExecutor): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true resource = Database result = Ok(()) saga_id = bc1ac890-e9e2-44d9-931e-3abe24cf75f5 saga_name = instance-start saga_node = MarkAsStarting
51292025-04-30T06:43:58.498ZTRACtest_utilization (SagaExecutor): loading roles actor = Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. } actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet saga_id = bc1ac890-e9e2-44d9-931e-3abe24cf75f5 saga_name = instance-start saga_node = MarkAsStarting
51302025-04-30T06:43:58.498ZTRACtest_utilization (SagaExecutor): authorize begin action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true resource = Database saga_id = bc1ac890-e9e2-44d9-931e-3abe24cf75f5 saga_name = instance-start saga_node = MarkAsStarting
51312025-04-30T06:43:58.498ZDEBGtest_utilization (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true roles = RoleSet { roles: {} } saga_id = bc1ac890-e9e2-44d9-931e-3abe24cf75f5 saga_name = instance-start saga_node = MarkAsStarting
51322025-04-30T06:43:58.499ZDEBGtest_utilization (SagaExecutor): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true resource = Database result = Ok(()) saga_id = bc1ac890-e9e2-44d9-931e-3abe24cf75f5 saga_name = instance-start saga_node = MarkAsStarting
51332025-04-30T06:43:58.526ZDEBGtest_utilization (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin"), (Silo, 001de000-5110-4000-8000-000000000000, "admin")} } saga_id = bc1ac890-e9e2-44d9-931e-3abe24cf75f5 saga_name = instance-start saga_node = MarkAsStarting
51342025-04-30T06:43:58.534ZDEBGtest_utilization (SagaExecutor): authorize result action = Read actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true resource = Instance { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000000, lookup_type: ById(001de000-5110-4000-8000-000000000000) }, key: 473b39f9-8669-45b3-b3d7-793f7e10fc31, lookup_type: ById(473b39f9-8669-45b3-b3d7-793f7e10fc31) }, key: d7bd2deb-ee27-4661-9e9a-524d734cd6ec, lookup_type: ById(d7bd2deb-ee27-4661-9e9a-524d734cd6ec) } result = Ok(()) saga_id = bc1ac890-e9e2-44d9-931e-3abe24cf75f5 saga_name = instance-start saga_node = MarkAsStarting
51352025-04-30T06:43:58.534ZTRACtest_utilization (SagaExecutor): authorize begin action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true resource = Database saga_id = bc1ac890-e9e2-44d9-931e-3abe24cf75f5 saga_name = instance-start saga_node = MarkAsStarting
51362025-04-30T06:43:58.534ZDEBGtest_utilization (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true roles = RoleSet { roles: {} } saga_id = bc1ac890-e9e2-44d9-931e-3abe24cf75f5 saga_name = instance-start saga_node = MarkAsStarting
51372025-04-30T06:43:58.535ZDEBGtest_utilization (SagaExecutor): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true resource = Database result = Ok(()) saga_id = bc1ac890-e9e2-44d9-931e-3abe24cf75f5 saga_name = instance-start saga_node = MarkAsStarting
51382025-04-30T06:43:58.565ZDEBGtest_utilization (SEC): saga log event new_state = N004 succeeded sec_id = ac55d8d9-a298-4534-898b-beee1c9e4e52
51392025-04-30T06:43:58.565ZDEBGtest_utilization (SecStore): recording saga event event_type = Succeeded(Object {"auto_restart": Object {"cooldown": Null, "policy": Null}, "boot_disk_id": Null, "hostname": String("the-host"), "identity": Object {"description": String("instance \\"utilization-test-instance\\""), "id": String("d7bd2deb-ee27-4661-9e9a-524d734cd6ec"), "name": String("utilization-test-instance"), "time_created": String("2025-04-30T06:43:54.249576Z"), "time_deleted": Null, "time_modified": String("2025-04-30T06:43:54.249576Z")}, "memory": Number(1073741824), "ncpus": Number(4), "project_id": String("473b39f9-8669-45b3-b3d7-793f7e10fc31"), "runtime_state": Object {"dst_propolis_id": Null, "gen": Number(3), "migration_id": Null, "nexus_state": String("Vmm"), "propolis_id": String("9e8dbed7-faa3-4634-886b-3a005dd8e99b"), "time_last_auto_restarted": Null, "time_updated": String("2025-04-30T06:43:54.249576Z")}, "updater_gen": Number(1), "updater_id": Null, "user_data": Array [Number(35), Number(99), Number(108), Number(111), Number(117), Number(100), Number(45), Number(99), Number(111), Number(110), Number(102), Number(105), Number(103), Number(10), Number(115), Number(121), Number(115), Number(116), Number(101), Number(109), Number(95), Number(105), Number(110), Number(102), Number(111), Number(58), Number(10), Number(32), Number(32), Number(100), Number(101), Number(102), Number(97), Number(117), Number(108), Number(116), Number(95), Number(117), Number(115), Number(101), Number(114), Number(58), Number(10), Number(32), Number(32), Number(32), Number(32), Number(110), Number(97), Number(109), Number(101), Number(58), Number(32), Number(111), Number(120), Number(105), Number(100), Number(101)]}) node_id = 4 saga_id = bc1ac890-e9e2-44d9-931e-3abe24cf75f5
51402025-04-30T06:43:58.584ZDEBGtest_utilization (SEC): saga log event new_state = N005 started sec_id = ac55d8d9-a298-4534-898b-beee1c9e4e52
51412025-04-30T06:43:58.584ZDEBGtest_utilization (SecStore): recording saga event event_type = Started node_id = 5 saga_id = bc1ac890-e9e2-44d9-931e-3abe24cf75f5
51422025-04-30T06:43:58.592ZINFOtest_utilization (SagaExecutor): start saga: ensuring instance dpd configuration instance_id = d7bd2deb-ee27-4661-9e9a-524d734cd6ec saga_id = bc1ac890-e9e2-44d9-931e-3abe24cf75f5 saga_name = instance-start start_reason = AutoStart
51432025-04-30T06:43:58.592ZTRACtest_utilization (InstanceAllocator): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000004ead, .. }) resource = Database
51442025-04-30T06:43:58.592ZDEBGtest_utilization (InstanceAllocator): roles roles = RoleSet { roles: {} }
51452025-04-30T06:43:58.593ZDEBGtest_utilization (InstanceAllocator): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000004ead, .. }) resource = Database result = Ok(())
51462025-04-30T06:43:58.602ZTRACtest_utilization (InstanceAllocator): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000004ead, .. }) resource = Sled { parent: Fleet, key: b6d65341-167c-41df-9b5c-41cded99c229, lookup_type: ById(b6d65341-167c-41df-9b5c-41cded99c229) }
51472025-04-30T06:43:58.602ZTRACtest_utilization (InstanceAllocator): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000004ead, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
51482025-04-30T06:43:58.602ZTRACtest_utilization (InstanceAllocator): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000004ead, .. }) resource = Database
51492025-04-30T06:43:58.602ZDEBGtest_utilization (InstanceAllocator): roles roles = RoleSet { roles: {} }
51502025-04-30T06:43:58.603ZDEBGtest_utilization (InstanceAllocator): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000004ead, .. }) resource = Database result = Ok(())
51512025-04-30T06:43:58.632ZDEBGtest_utilization (InstanceAllocator): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "viewer")} }
51522025-04-30T06:43:58.638ZDEBGtest_utilization (InstanceAllocator): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000004ead, .. }) resource = Sled { parent: Fleet, key: b6d65341-167c-41df-9b5c-41cded99c229, lookup_type: ById(b6d65341-167c-41df-9b5c-41cded99c229) } result = Ok(())
51532025-04-30T06:43:58.638ZINFOtest_utilization (nexus): looking up instance's primary network interface instance_id = d7bd2deb-ee27-4661-9e9a-524d734cd6ec
51542025-04-30T06:43:58.638ZTRACtest_utilization (SagaExecutor): authorize begin action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true resource = Database saga_id = bc1ac890-e9e2-44d9-931e-3abe24cf75f5 saga_name = instance-start saga_node = DpdEnsure
51552025-04-30T06:43:58.638ZDEBGtest_utilization (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true roles = RoleSet { roles: {} } saga_id = bc1ac890-e9e2-44d9-931e-3abe24cf75f5 saga_name = instance-start saga_node = DpdEnsure
51562025-04-30T06:43:58.638ZDEBGtest_utilization (SagaExecutor): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true resource = Database result = Ok(()) saga_id = bc1ac890-e9e2-44d9-931e-3abe24cf75f5 saga_name = instance-start saga_node = DpdEnsure
51572025-04-30T06:43:58.658ZTRACtest_utilization (SagaExecutor): authorize begin action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true resource = Database saga_id = bc1ac890-e9e2-44d9-931e-3abe24cf75f5 saga_name = instance-start saga_node = DpdEnsure
51582025-04-30T06:43:58.658ZDEBGtest_utilization (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true roles = RoleSet { roles: {} } saga_id = bc1ac890-e9e2-44d9-931e-3abe24cf75f5 saga_name = instance-start saga_node = DpdEnsure
51592025-04-30T06:43:58.659ZDEBGtest_utilization (SagaExecutor): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true resource = Database result = Ok(()) saga_id = bc1ac890-e9e2-44d9-931e-3abe24cf75f5 saga_name = instance-start saga_node = DpdEnsure
51602025-04-30T06:43:58.664ZTRACtest_utilization (SagaExecutor): authorize begin action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true resource = Database saga_id = bc1ac890-e9e2-44d9-931e-3abe24cf75f5 saga_name = instance-start saga_node = DpdEnsure
51612025-04-30T06:43:58.664ZDEBGtest_utilization (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true roles = RoleSet { roles: {} } saga_id = bc1ac890-e9e2-44d9-931e-3abe24cf75f5 saga_name = instance-start saga_node = DpdEnsure
51622025-04-30T06:43:58.664ZDEBGtest_utilization (SagaExecutor): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true resource = Database result = Ok(()) saga_id = bc1ac890-e9e2-44d9-931e-3abe24cf75f5 saga_name = instance-start saga_node = DpdEnsure
51632025-04-30T06:43:58.670ZTRACtest_utilization (SagaExecutor): authorize begin action = ListChildren actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true resource = Instance { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000000, lookup_type: ById(001de000-5110-4000-8000-000000000000) }, key: 473b39f9-8669-45b3-b3d7-793f7e10fc31, lookup_type: ById(473b39f9-8669-45b3-b3d7-793f7e10fc31) }, key: d7bd2deb-ee27-4661-9e9a-524d734cd6ec, lookup_type: ById(d7bd2deb-ee27-4661-9e9a-524d734cd6ec) } saga_id = bc1ac890-e9e2-44d9-931e-3abe24cf75f5 saga_name = instance-start saga_node = DpdEnsure
51642025-04-30T06:43:58.670ZTRACtest_utilization (SagaExecutor): loading roles actor = Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. } actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true resource_id = 473b39f9-8669-45b3-b3d7-793f7e10fc31 resource_type = Project saga_id = bc1ac890-e9e2-44d9-931e-3abe24cf75f5 saga_name = instance-start saga_node = DpdEnsure
51652025-04-30T06:43:58.670ZTRACtest_utilization (SagaExecutor): authorize begin action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true resource = Database saga_id = bc1ac890-e9e2-44d9-931e-3abe24cf75f5 saga_name = instance-start saga_node = DpdEnsure
51662025-04-30T06:43:58.670ZDEBGtest_utilization (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true roles = RoleSet { roles: {} } saga_id = bc1ac890-e9e2-44d9-931e-3abe24cf75f5 saga_name = instance-start saga_node = DpdEnsure
51672025-04-30T06:43:58.671ZDEBGtest_utilization (SagaExecutor): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true resource = Database result = Ok(()) saga_id = bc1ac890-e9e2-44d9-931e-3abe24cf75f5 saga_name = instance-start saga_node = DpdEnsure
51682025-04-30T06:43:58.675ZTRACtest_utilization (SagaExecutor): loading roles actor = Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. } actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true resource_id = 001de000-5110-4000-8000-000000000000 resource_type = Silo saga_id = bc1ac890-e9e2-44d9-931e-3abe24cf75f5 saga_name = instance-start saga_node = DpdEnsure
51692025-04-30T06:43:58.675ZTRACtest_utilization (SagaExecutor): authorize begin action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true resource = Database saga_id = bc1ac890-e9e2-44d9-931e-3abe24cf75f5 saga_name = instance-start saga_node = DpdEnsure
51702025-04-30T06:43:58.675ZDEBGtest_utilization (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true roles = RoleSet { roles: {} } saga_id = bc1ac890-e9e2-44d9-931e-3abe24cf75f5 saga_name = instance-start saga_node = DpdEnsure
51712025-04-30T06:43:58.675ZDEBGtest_utilization (SagaExecutor): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true resource = Database result = Ok(()) saga_id = bc1ac890-e9e2-44d9-931e-3abe24cf75f5 saga_name = instance-start saga_node = DpdEnsure
51722025-04-30T06:43:58.679ZTRACtest_utilization (SagaExecutor): loading roles actor = Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. } actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet saga_id = bc1ac890-e9e2-44d9-931e-3abe24cf75f5 saga_name = instance-start saga_node = DpdEnsure
51732025-04-30T06:43:58.679ZTRACtest_utilization (SagaExecutor): authorize begin action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true resource = Database saga_id = bc1ac890-e9e2-44d9-931e-3abe24cf75f5 saga_name = instance-start saga_node = DpdEnsure
51742025-04-30T06:43:58.679ZDEBGtest_utilization (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true roles = RoleSet { roles: {} } saga_id = bc1ac890-e9e2-44d9-931e-3abe24cf75f5 saga_name = instance-start saga_node = DpdEnsure
51752025-04-30T06:43:58.681ZDEBGtest_utilization (SagaExecutor): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true resource = Database result = Ok(()) saga_id = bc1ac890-e9e2-44d9-931e-3abe24cf75f5 saga_name = instance-start saga_node = DpdEnsure
51762025-04-30T06:43:58.692ZDEBGtest_utilization (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin"), (Silo, 001de000-5110-4000-8000-000000000000, "admin")} } saga_id = bc1ac890-e9e2-44d9-931e-3abe24cf75f5 saga_name = instance-start saga_node = DpdEnsure
51772025-04-30T06:43:58.703ZDEBGtest_utilization (SagaExecutor): authorize result action = ListChildren actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true resource = Instance { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000000, lookup_type: ById(001de000-5110-4000-8000-000000000000) }, key: 473b39f9-8669-45b3-b3d7-793f7e10fc31, lookup_type: ById(473b39f9-8669-45b3-b3d7-793f7e10fc31) }, key: d7bd2deb-ee27-4661-9e9a-524d734cd6ec, lookup_type: ById(d7bd2deb-ee27-4661-9e9a-524d734cd6ec) } result = Ok(()) saga_id = bc1ac890-e9e2-44d9-931e-3abe24cf75f5 saga_name = instance-start saga_node = DpdEnsure
51782025-04-30T06:43:58.703ZTRACtest_utilization (SagaExecutor): authorize begin action = ListChildren actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true resource = Instance { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000000, lookup_type: ById(001de000-5110-4000-8000-000000000000) }, key: 473b39f9-8669-45b3-b3d7-793f7e10fc31, lookup_type: ById(473b39f9-8669-45b3-b3d7-793f7e10fc31) }, key: d7bd2deb-ee27-4661-9e9a-524d734cd6ec, lookup_type: ById(d7bd2deb-ee27-4661-9e9a-524d734cd6ec) } saga_id = bc1ac890-e9e2-44d9-931e-3abe24cf75f5 saga_name = instance-start saga_node = DpdEnsure
51792025-04-30T06:43:58.703ZTRACtest_utilization (SagaExecutor): loading roles actor = Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. } actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true resource_id = 473b39f9-8669-45b3-b3d7-793f7e10fc31 resource_type = Project saga_id = bc1ac890-e9e2-44d9-931e-3abe24cf75f5 saga_name = instance-start saga_node = DpdEnsure
51802025-04-30T06:43:58.703ZTRACtest_utilization (SagaExecutor): authorize begin action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true resource = Database saga_id = bc1ac890-e9e2-44d9-931e-3abe24cf75f5 saga_name = instance-start saga_node = DpdEnsure
51812025-04-30T06:43:58.703ZDEBGtest_utilization (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true roles = RoleSet { roles: {} } saga_id = bc1ac890-e9e2-44d9-931e-3abe24cf75f5 saga_name = instance-start saga_node = DpdEnsure
51822025-04-30T06:43:58.704ZDEBGtest_utilization (SagaExecutor): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true resource = Database result = Ok(()) saga_id = bc1ac890-e9e2-44d9-931e-3abe24cf75f5 saga_name = instance-start saga_node = DpdEnsure
51832025-04-30T06:43:58.711ZTRACtest_utilization (SagaExecutor): loading roles actor = Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. } actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true resource_id = 001de000-5110-4000-8000-000000000000 resource_type = Silo saga_id = bc1ac890-e9e2-44d9-931e-3abe24cf75f5 saga_name = instance-start saga_node = DpdEnsure
51842025-04-30T06:43:58.711ZTRACtest_utilization (SagaExecutor): authorize begin action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true resource = Database saga_id = bc1ac890-e9e2-44d9-931e-3abe24cf75f5 saga_name = instance-start saga_node = DpdEnsure
51852025-04-30T06:43:58.711ZDEBGtest_utilization (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true roles = RoleSet { roles: {} } saga_id = bc1ac890-e9e2-44d9-931e-3abe24cf75f5 saga_name = instance-start saga_node = DpdEnsure
51862025-04-30T06:43:58.711ZDEBGtest_utilization (SagaExecutor): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true resource = Database result = Ok(()) saga_id = bc1ac890-e9e2-44d9-931e-3abe24cf75f5 saga_name = instance-start saga_node = DpdEnsure
51872025-04-30T06:43:58.716ZTRACtest_utilization (SagaExecutor): loading roles actor = Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. } actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet saga_id = bc1ac890-e9e2-44d9-931e-3abe24cf75f5 saga_name = instance-start saga_node = DpdEnsure
51882025-04-30T06:43:58.716ZTRACtest_utilization (SagaExecutor): authorize begin action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true resource = Database saga_id = bc1ac890-e9e2-44d9-931e-3abe24cf75f5 saga_name = instance-start saga_node = DpdEnsure
51892025-04-30T06:43:58.716ZDEBGtest_utilization (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true roles = RoleSet { roles: {} } saga_id = bc1ac890-e9e2-44d9-931e-3abe24cf75f5 saga_name = instance-start saga_node = DpdEnsure
51902025-04-30T06:43:58.716ZDEBGtest_utilization (SagaExecutor): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true resource = Database result = Ok(()) saga_id = bc1ac890-e9e2-44d9-931e-3abe24cf75f5 saga_name = instance-start saga_node = DpdEnsure
51912025-04-30T06:43:58.720ZDEBGtest_utilization (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin"), (Silo, 001de000-5110-4000-8000-000000000000, "admin")} } saga_id = bc1ac890-e9e2-44d9-931e-3abe24cf75f5 saga_name = instance-start saga_node = DpdEnsure
51922025-04-30T06:43:58.729ZDEBGtest_utilization (SagaExecutor): authorize result action = ListChildren actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true resource = Instance { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000000, lookup_type: ById(001de000-5110-4000-8000-000000000000) }, key: 473b39f9-8669-45b3-b3d7-793f7e10fc31, lookup_type: ById(473b39f9-8669-45b3-b3d7-793f7e10fc31) }, key: d7bd2deb-ee27-4661-9e9a-524d734cd6ec, lookup_type: ById(d7bd2deb-ee27-4661-9e9a-524d734cd6ec) } result = Ok(()) saga_id = bc1ac890-e9e2-44d9-931e-3abe24cf75f5 saga_name = instance-start saga_node = DpdEnsure
51932025-04-30T06:43:58.730ZTRACtest_utilization (SagaExecutor): authorize begin action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true resource = Database saga_id = bc1ac890-e9e2-44d9-931e-3abe24cf75f5 saga_name = instance-start saga_node = DpdEnsure
51942025-04-30T06:43:58.730ZDEBGtest_utilization (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true roles = RoleSet { roles: {} } saga_id = bc1ac890-e9e2-44d9-931e-3abe24cf75f5 saga_name = instance-start saga_node = DpdEnsure
51952025-04-30T06:43:58.730ZDEBGtest_utilization (SagaExecutor): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true resource = Database result = Ok(()) saga_id = bc1ac890-e9e2-44d9-931e-3abe24cf75f5 saga_name = instance-start saga_node = DpdEnsure
51962025-04-30T06:43:58.744ZINFOtest_utilization (nexus): looking up instance's external IPs instance_id = d7bd2deb-ee27-4661-9e9a-524d734cd6ec
51972025-04-30T06:43:58.745ZTRACtest_utilization (SagaExecutor): authorize begin action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true resource = Database saga_id = bc1ac890-e9e2-44d9-931e-3abe24cf75f5 saga_name = instance-start saga_node = DpdEnsure
51982025-04-30T06:43:58.745ZDEBGtest_utilization (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true roles = RoleSet { roles: {} } saga_id = bc1ac890-e9e2-44d9-931e-3abe24cf75f5 saga_name = instance-start saga_node = DpdEnsure
51992025-04-30T06:43:58.752ZDEBGtest_utilization (SagaExecutor): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true resource = Database result = Ok(()) saga_id = bc1ac890-e9e2-44d9-931e-3abe24cf75f5 saga_name = instance-start saga_node = DpdEnsure
52002025-04-30T06:43:58.757ZTRACtest_utilization (SagaExecutor): authorize begin action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true resource = Database saga_id = bc1ac890-e9e2-44d9-931e-3abe24cf75f5 saga_name = instance-start saga_node = DpdEnsure
52012025-04-30T06:43:58.757ZDEBGtest_utilization (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true roles = RoleSet { roles: {} } saga_id = bc1ac890-e9e2-44d9-931e-3abe24cf75f5 saga_name = instance-start saga_node = DpdEnsure
52022025-04-30T06:43:58.758ZDEBGtest_utilization (SagaExecutor): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true resource = Database result = Ok(()) saga_id = bc1ac890-e9e2-44d9-931e-3abe24cf75f5 saga_name = instance-start saga_node = DpdEnsure
52032025-04-30T06:43:58.768ZTRACtest_utilization (InstanceAllocator): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000004ead, .. }) resource = Fleet
52042025-04-30T06:43:58.768ZTRACtest_utilization (InstanceAllocator): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000004ead, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
52052025-04-30T06:43:58.768ZTRACtest_utilization (InstanceAllocator): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000004ead, .. }) resource = Database
52062025-04-30T06:43:58.768ZDEBGtest_utilization (InstanceAllocator): roles roles = RoleSet { roles: {} }
52072025-04-30T06:43:58.769ZDEBGtest_utilization (InstanceAllocator): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000004ead, .. }) resource = Database result = Ok(())
52082025-04-30T06:43:58.772ZDEBGtest_utilization (InstanceAllocator): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "viewer")} }
52092025-04-30T06:43:58.774ZDEBGtest_utilization (InstanceAllocator): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000004ead, .. }) resource = Fleet result = Ok(())
52102025-04-30T06:43:58.774ZTRACtest_utilization (InstanceAllocator): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000004ead, .. }) resource = Database
52112025-04-30T06:43:58.774ZDEBGtest_utilization (InstanceAllocator): roles roles = RoleSet { roles: {} }
52122025-04-30T06:43:58.775ZDEBGtest_utilization (InstanceAllocator): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000004ead, .. }) resource = Database result = Ok(())
52132025-04-30T06:43:58.779ZDEBGtest_utilization (SEC): saga log event new_state = N005 succeeded sec_id = ac55d8d9-a298-4534-898b-beee1c9e4e52
52142025-04-30T06:43:58.779ZDEBGtest_utilization (SecStore): recording saga event event_type = Succeeded(Null) node_id = 5 saga_id = bc1ac890-e9e2-44d9-931e-3abe24cf75f5
52152025-04-30T06:43:58.782ZDEBGtest_utilization (SEC): saga log event new_state = N006 started sec_id = ac55d8d9-a298-4534-898b-beee1c9e4e52
52162025-04-30T06:43:58.782ZDEBGtest_utilization (SecStore): recording saga event event_type = Started node_id = 6 saga_id = bc1ac890-e9e2-44d9-931e-3abe24cf75f5
52172025-04-30T06:43:58.787ZDEBGtest_utilization (BackgroundTasks): activating background_task = v2p_manager iteration = 2 reason = Signaled
52182025-04-30T06:43:58.787ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database
52192025-04-30T06:43:58.787ZDEBGtest_utilization (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
52202025-04-30T06:43:58.787ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
52212025-04-30T06:43:58.787ZDEBGtest_utilization (SEC): saga log event new_state = N006 succeeded sec_id = ac55d8d9-a298-4534-898b-beee1c9e4e52
52222025-04-30T06:43:58.787ZDEBGtest_utilization (SecStore): recording saga event event_type = Succeeded(Null) node_id = 6 saga_id = bc1ac890-e9e2-44d9-931e-3abe24cf75f5
52232025-04-30T06:43:58.852ZDEBGtest_utilization (SEC): saga log event new_state = N007 started sec_id = ac55d8d9-a298-4534-898b-beee1c9e4e52
52242025-04-30T06:43:58.852ZDEBGtest_utilization (SecStore): recording saga event event_type = Started node_id = 7 saga_id = bc1ac890-e9e2-44d9-931e-3abe24cf75f5
52252025-04-30T06:43:58.853ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database
52262025-04-30T06:43:58.853ZDEBGtest_utilization (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
52272025-04-30T06:43:58.854ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
52282025-04-30T06:43:58.867ZINFOtest_utilization (SagaExecutor): start saga: ensuring instance is registered on sled instance_id = d7bd2deb-ee27-4661-9e9a-524d734cd6ec saga_id = bc1ac890-e9e2-44d9-931e-3abe24cf75f5 saga_name = instance-start sled_id = b6d65341-167c-41df-9b5c-41cded99c229 start_reason = AutoStart
52292025-04-30T06:43:58.867ZTRACtest_utilization (SagaExecutor): authorize begin action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true resource = Database saga_id = bc1ac890-e9e2-44d9-931e-3abe24cf75f5 saga_name = instance-start saga_node = EnsureRegistered
52302025-04-30T06:43:58.867ZDEBGtest_utilization (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true roles = RoleSet { roles: {} } saga_id = bc1ac890-e9e2-44d9-931e-3abe24cf75f5 saga_name = instance-start saga_node = EnsureRegistered
52312025-04-30T06:43:58.867ZDEBGtest_utilization (SagaExecutor): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true resource = Database result = Ok(()) saga_id = bc1ac890-e9e2-44d9-931e-3abe24cf75f5 saga_name = instance-start saga_node = EnsureRegistered
52322025-04-30T06:43:58.872ZTRACtest_utilization (SagaExecutor): authorize begin action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true resource = Database saga_id = bc1ac890-e9e2-44d9-931e-3abe24cf75f5 saga_name = instance-start saga_node = EnsureRegistered
52332025-04-30T06:43:58.872ZDEBGtest_utilization (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true roles = RoleSet { roles: {} } saga_id = bc1ac890-e9e2-44d9-931e-3abe24cf75f5 saga_name = instance-start saga_node = EnsureRegistered
52342025-04-30T06:43:58.872ZDEBGtest_utilization (SagaExecutor): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true resource = Database result = Ok(()) saga_id = bc1ac890-e9e2-44d9-931e-3abe24cf75f5 saga_name = instance-start saga_node = EnsureRegistered
52352025-04-30T06:43:58.875ZTRACtest_utilization (SagaExecutor): authorize begin action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true resource = Database saga_id = bc1ac890-e9e2-44d9-931e-3abe24cf75f5 saga_name = instance-start saga_node = EnsureRegistered
52362025-04-30T06:43:58.875ZDEBGtest_utilization (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true roles = RoleSet { roles: {} } saga_id = bc1ac890-e9e2-44d9-931e-3abe24cf75f5 saga_name = instance-start saga_node = EnsureRegistered
52372025-04-30T06:43:58.875ZDEBGtest_utilization (SagaExecutor): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true resource = Database result = Ok(()) saga_id = bc1ac890-e9e2-44d9-931e-3abe24cf75f5 saga_name = instance-start saga_node = EnsureRegistered
52382025-04-30T06:43:58.877ZTRACtest_utilization (BackgroundTasks): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet
52392025-04-30T06:43:58.877ZTRACtest_utilization (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = v2p_manager resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
52402025-04-30T06:43:58.877ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database
52412025-04-30T06:43:58.877ZDEBGtest_utilization (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
52422025-04-30T06:43:58.878ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
52432025-04-30T06:43:58.878ZTRACtest_utilization (SagaExecutor): authorize begin action = Modify actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true resource = Instance { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000000, lookup_type: ById(001de000-5110-4000-8000-000000000000) }, key: 473b39f9-8669-45b3-b3d7-793f7e10fc31, lookup_type: ById(473b39f9-8669-45b3-b3d7-793f7e10fc31) }, key: d7bd2deb-ee27-4661-9e9a-524d734cd6ec, lookup_type: ById(d7bd2deb-ee27-4661-9e9a-524d734cd6ec) } saga_id = bc1ac890-e9e2-44d9-931e-3abe24cf75f5 saga_name = instance-start saga_node = EnsureRegistered
52442025-04-30T06:43:58.878ZTRACtest_utilization (SagaExecutor): loading roles actor = Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. } actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true resource_id = 473b39f9-8669-45b3-b3d7-793f7e10fc31 resource_type = Project saga_id = bc1ac890-e9e2-44d9-931e-3abe24cf75f5 saga_name = instance-start saga_node = EnsureRegistered
52452025-04-30T06:43:58.878ZTRACtest_utilization (SagaExecutor): authorize begin action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true resource = Database saga_id = bc1ac890-e9e2-44d9-931e-3abe24cf75f5 saga_name = instance-start saga_node = EnsureRegistered
52462025-04-30T06:43:58.878ZDEBGtest_utilization (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true roles = RoleSet { roles: {} } saga_id = bc1ac890-e9e2-44d9-931e-3abe24cf75f5 saga_name = instance-start saga_node = EnsureRegistered
52472025-04-30T06:43:58.879ZDEBGtest_utilization (SagaExecutor): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true resource = Database result = Ok(()) saga_id = bc1ac890-e9e2-44d9-931e-3abe24cf75f5 saga_name = instance-start saga_node = EnsureRegistered
52482025-04-30T06:43:58.881ZDEBGtest_utilization (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
52492025-04-30T06:43:58.884ZDEBGtest_utilization (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
52502025-04-30T06:43:58.884ZTRACtest_utilization (BackgroundTasks): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet
52512025-04-30T06:43:58.884ZTRACtest_utilization (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = v2p_manager resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
52522025-04-30T06:43:58.884ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database
52532025-04-30T06:43:58.884ZDEBGtest_utilization (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
52542025-04-30T06:43:58.884ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
52552025-04-30T06:43:58.884ZTRACtest_utilization (SagaExecutor): loading roles actor = Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. } actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true resource_id = 001de000-5110-4000-8000-000000000000 resource_type = Silo saga_id = bc1ac890-e9e2-44d9-931e-3abe24cf75f5 saga_name = instance-start saga_node = EnsureRegistered
52562025-04-30T06:43:58.885ZTRACtest_utilization (SagaExecutor): authorize begin action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true resource = Database saga_id = bc1ac890-e9e2-44d9-931e-3abe24cf75f5 saga_name = instance-start saga_node = EnsureRegistered
52572025-04-30T06:43:58.885ZDEBGtest_utilization (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true roles = RoleSet { roles: {} } saga_id = bc1ac890-e9e2-44d9-931e-3abe24cf75f5 saga_name = instance-start saga_node = EnsureRegistered
52582025-04-30T06:43:58.894ZDEBGtest_utilization (SagaExecutor): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true resource = Database result = Ok(()) saga_id = bc1ac890-e9e2-44d9-931e-3abe24cf75f5 saga_name = instance-start saga_node = EnsureRegistered
52592025-04-30T06:43:58.897ZDEBGtest_utilization (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
52602025-04-30T06:43:58.901ZDEBGtest_utilization (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
52612025-04-30T06:43:58.901ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database
52622025-04-30T06:43:58.901ZDEBGtest_utilization (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
52632025-04-30T06:43:58.901ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
52642025-04-30T06:43:58.905ZTRACtest_utilization (SagaExecutor): loading roles actor = Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. } actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet saga_id = bc1ac890-e9e2-44d9-931e-3abe24cf75f5 saga_name = instance-start saga_node = EnsureRegistered
52652025-04-30T06:43:58.905ZTRACtest_utilization (SagaExecutor): authorize begin action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true resource = Database saga_id = bc1ac890-e9e2-44d9-931e-3abe24cf75f5 saga_name = instance-start saga_node = EnsureRegistered
52662025-04-30T06:43:58.905ZDEBGtest_utilization (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true roles = RoleSet { roles: {} } saga_id = bc1ac890-e9e2-44d9-931e-3abe24cf75f5 saga_name = instance-start saga_node = EnsureRegistered
52672025-04-30T06:43:58.906ZDEBGtest_utilization (SagaExecutor): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true resource = Database result = Ok(()) saga_id = bc1ac890-e9e2-44d9-931e-3abe24cf75f5 saga_name = instance-start saga_node = EnsureRegistered
52682025-04-30T06:43:58.911ZDEBGtest_utilization (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin"), (Silo, 001de000-5110-4000-8000-000000000000, "admin")} } saga_id = bc1ac890-e9e2-44d9-931e-3abe24cf75f5 saga_name = instance-start saga_node = EnsureRegistered
52692025-04-30T06:43:58.920ZDEBGtest_utilization (SagaExecutor): authorize result action = Modify actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true resource = Instance { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000000, lookup_type: ById(001de000-5110-4000-8000-000000000000) }, key: 473b39f9-8669-45b3-b3d7-793f7e10fc31, lookup_type: ById(473b39f9-8669-45b3-b3d7-793f7e10fc31) }, key: d7bd2deb-ee27-4661-9e9a-524d734cd6ec, lookup_type: ById(d7bd2deb-ee27-4661-9e9a-524d734cd6ec) } result = Ok(()) saga_id = bc1ac890-e9e2-44d9-931e-3abe24cf75f5 saga_name = instance-start saga_node = EnsureRegistered
52702025-04-30T06:43:58.920ZTRACtest_utilization (SagaExecutor): authorize begin action = Modify actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true resource = Instance { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000000, lookup_type: ById(001de000-5110-4000-8000-000000000000) }, key: 473b39f9-8669-45b3-b3d7-793f7e10fc31, lookup_type: ById(473b39f9-8669-45b3-b3d7-793f7e10fc31) }, key: d7bd2deb-ee27-4661-9e9a-524d734cd6ec, lookup_type: ById(d7bd2deb-ee27-4661-9e9a-524d734cd6ec) } saga_id = bc1ac890-e9e2-44d9-931e-3abe24cf75f5 saga_name = instance-start saga_node = EnsureRegistered
52712025-04-30T06:43:58.920ZTRACtest_utilization (SagaExecutor): loading roles actor = Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. } actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true resource_id = 473b39f9-8669-45b3-b3d7-793f7e10fc31 resource_type = Project saga_id = bc1ac890-e9e2-44d9-931e-3abe24cf75f5 saga_name = instance-start saga_node = EnsureRegistered
52722025-04-30T06:43:58.920ZTRACtest_utilization (SagaExecutor): authorize begin action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true resource = Database saga_id = bc1ac890-e9e2-44d9-931e-3abe24cf75f5 saga_name = instance-start saga_node = EnsureRegistered
52732025-04-30T06:43:58.920ZDEBGtest_utilization (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true roles = RoleSet { roles: {} } saga_id = bc1ac890-e9e2-44d9-931e-3abe24cf75f5 saga_name = instance-start saga_node = EnsureRegistered
52742025-04-30T06:43:58.923ZDEBGtest_utilization (SagaExecutor): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true resource = Database result = Ok(()) saga_id = bc1ac890-e9e2-44d9-931e-3abe24cf75f5 saga_name = instance-start saga_node = EnsureRegistered
52752025-04-30T06:43:58.975ZTRACtest_utilization (SagaExecutor): loading roles actor = Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. } actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true resource_id = 001de000-5110-4000-8000-000000000000 resource_type = Silo saga_id = bc1ac890-e9e2-44d9-931e-3abe24cf75f5 saga_name = instance-start saga_node = EnsureRegistered
52762025-04-30T06:43:58.975ZTRACtest_utilization (SagaExecutor): authorize begin action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true resource = Database saga_id = bc1ac890-e9e2-44d9-931e-3abe24cf75f5 saga_name = instance-start saga_node = EnsureRegistered
52772025-04-30T06:43:58.976ZDEBGtest_utilization (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true roles = RoleSet { roles: {} } saga_id = bc1ac890-e9e2-44d9-931e-3abe24cf75f5 saga_name = instance-start saga_node = EnsureRegistered
52782025-04-30T06:43:58.977ZDEBGtest_utilization (SagaExecutor): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true resource = Database result = Ok(()) saga_id = bc1ac890-e9e2-44d9-931e-3abe24cf75f5 saga_name = instance-start saga_node = EnsureRegistered
52792025-04-30T06:43:59.036ZDEBGtest_utilization (BackgroundTasks): client request SledAgent = b6d65341-167c-41df-9b5c-41cded99c229 background_task = v2p_manager body = None method = GET uri = http://[::1]:39365/v2p
52802025-04-30T06:43:59.041ZTRACtest_utilization (SagaExecutor): loading roles actor = Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. } actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet saga_id = bc1ac890-e9e2-44d9-931e-3abe24cf75f5 saga_name = instance-start saga_node = EnsureRegistered
52812025-04-30T06:43:59.041ZTRACtest_utilization (SagaExecutor): authorize begin action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true resource = Database saga_id = bc1ac890-e9e2-44d9-931e-3abe24cf75f5 saga_name = instance-start saga_node = EnsureRegistered
52822025-04-30T06:43:59.041ZDEBGtest_utilization (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true roles = RoleSet { roles: {} } saga_id = bc1ac890-e9e2-44d9-931e-3abe24cf75f5 saga_name = instance-start saga_node = EnsureRegistered
52832025-04-30T06:43:59.042ZDEBGtest_utilization (SagaExecutor): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true resource = Database result = Ok(()) saga_id = bc1ac890-e9e2-44d9-931e-3abe24cf75f5 saga_name = instance-start saga_node = EnsureRegistered
52842025-04-30T06:43:59.042ZINFOtest_utilization (dropshot): accepted connection local_addr = [::1]:39365 remote_addr = [::1]:52234 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
52852025-04-30T06:43:59.043ZDEBGtest_utilization (collection-task): sent timer-based collection request to the collection task collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
52862025-04-30T06:43:59.043ZDEBGtest_utilization (collection-task): collecting from producer collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
52872025-04-30T06:43:59.044ZTRACtest_utilization (dropshot): incoming request local_addr = [::1]:39365 method = GET remote_addr = [::1]:52234 req_id = 2a87887d-7df8-4de9-b6ba-0254140592b5 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /v2p
52882025-04-30T06:43:59.045ZINFOtest_utilization (dropshot): request completed latency_us = 1117 local_addr = [::1]:39365 method = GET remote_addr = [::1]:52234 req_id = 2a87887d-7df8-4de9-b6ba-0254140592b5 response_code = 200 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /v2p
52892025-04-30T06:43:59.045ZTRACtest_utilization (collection-task): sent collection request to producer collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
52902025-04-30T06:43:59.046ZDEBGtest_utilization (collection-task): collected results from producer collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 n_results = 1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
52912025-04-30T06:43:59.046ZTRACtest_utilization (collection-task): forwarded results to main collection loop collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
52922025-04-30T06:43:59.046ZTRACtest_utilization (collection-task): top of inner collection loop, waiting for next request collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
52932025-04-30T06:43:59.046ZDEBGtest_utilization (BackgroundTasks): client response SledAgent = b6d65341-167c-41df-9b5c-41cded99c229 background_task = v2p_manager result = Ok(Response { url: "http://[::1]:39365/v2p", status: 200, headers: {"content-type": "application/json", "x-request-id": "2a87887d-7df8-4de9-b6ba-0254140592b5", "content-length": "2", "date": "Wed, 30 Apr 2025 06:43:59 GMT"} })
52942025-04-30T06:43:59.046ZINFOtest_utilization (BackgroundTasks): found opte v2p mappings background_task = v2p_manager interfaces = {} sled = sim-b6d65341
52952025-04-30T06:43:59.046ZINFOtest_utilization (BackgroundTasks): v2p mappings to delete background_task = v2p_manager mappings = [] sled = sim-b6d65341
52962025-04-30T06:43:59.046ZINFOtest_utilization (BackgroundTasks): v2p mappings to add background_task = v2p_manager mappings = [VirtualNetworkInterfaceHost { physical_host_ip: ::1, virtual_ip: 172.30.0.5, virtual_mac: MacAddr(MacAddr6([168, 64, 37, 240, 0, 0])), vni: Vni(11911937) }] sled = sim-b6d65341
52972025-04-30T06:43:59.046ZDEBGtest_utilization (BackgroundTasks): client request SledAgent = b6d65341-167c-41df-9b5c-41cded99c229 background_task = v2p_manager body = Some(Body) method = PUT uri = http://[::1]:39365/v2p
52982025-04-30T06:43:59.046ZDEBGtest_utilization (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin"), (Silo, 001de000-5110-4000-8000-000000000000, "admin")} } saga_id = bc1ac890-e9e2-44d9-931e-3abe24cf75f5 saga_name = instance-start saga_node = EnsureRegistered
52992025-04-30T06:43:59.054ZDEBGtest_utilization (SagaExecutor): authorize result action = Modify actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true resource = Instance { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000000, lookup_type: ById(001de000-5110-4000-8000-000000000000) }, key: 473b39f9-8669-45b3-b3d7-793f7e10fc31, lookup_type: ById(473b39f9-8669-45b3-b3d7-793f7e10fc31) }, key: d7bd2deb-ee27-4661-9e9a-524d734cd6ec, lookup_type: ById(d7bd2deb-ee27-4661-9e9a-524d734cd6ec) } result = Ok(()) saga_id = bc1ac890-e9e2-44d9-931e-3abe24cf75f5 saga_name = instance-start saga_node = EnsureRegistered
53002025-04-30T06:43:59.055ZTRACtest_utilization (SagaExecutor): authorize begin action = ListChildren actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true resource = Instance { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000000, lookup_type: ById(001de000-5110-4000-8000-000000000000) }, key: 473b39f9-8669-45b3-b3d7-793f7e10fc31, lookup_type: ById(473b39f9-8669-45b3-b3d7-793f7e10fc31) }, key: d7bd2deb-ee27-4661-9e9a-524d734cd6ec, lookup_type: ById(d7bd2deb-ee27-4661-9e9a-524d734cd6ec) } saga_id = bc1ac890-e9e2-44d9-931e-3abe24cf75f5 saga_name = instance-start saga_node = EnsureRegistered
53012025-04-30T06:43:59.055ZTRACtest_utilization (SagaExecutor): loading roles actor = Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. } actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true resource_id = 473b39f9-8669-45b3-b3d7-793f7e10fc31 resource_type = Project saga_id = bc1ac890-e9e2-44d9-931e-3abe24cf75f5 saga_name = instance-start saga_node = EnsureRegistered
53022025-04-30T06:43:59.055ZTRACtest_utilization (SagaExecutor): authorize begin action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true resource = Database saga_id = bc1ac890-e9e2-44d9-931e-3abe24cf75f5 saga_name = instance-start saga_node = EnsureRegistered
53032025-04-30T06:43:59.055ZDEBGtest_utilization (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true roles = RoleSet { roles: {} } saga_id = bc1ac890-e9e2-44d9-931e-3abe24cf75f5 saga_name = instance-start saga_node = EnsureRegistered
53042025-04-30T06:43:59.056ZDEBGtest_utilization (SagaExecutor): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true resource = Database result = Ok(()) saga_id = bc1ac890-e9e2-44d9-931e-3abe24cf75f5 saga_name = instance-start saga_node = EnsureRegistered
53052025-04-30T06:43:59.056ZTRACtest_utilization (dropshot): incoming request local_addr = [::1]:39365 method = PUT remote_addr = [::1]:52234 req_id = d5c36ca6-7eef-41b3-b790-fccac7015ebf sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /v2p
53062025-04-30T06:43:59.057ZINFOtest_utilization (dropshot): request completed latency_us = 438 local_addr = [::1]:39365 method = PUT remote_addr = [::1]:52234 req_id = d5c36ca6-7eef-41b3-b790-fccac7015ebf response_code = 204 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /v2p
53072025-04-30T06:43:59.057ZDEBGtest_utilization (BackgroundTasks): client response SledAgent = b6d65341-167c-41df-9b5c-41cded99c229 background_task = v2p_manager result = Ok(Response { url: "http://[::1]:39365/v2p", status: 204, headers: {"x-request-id": "d5c36ca6-7eef-41b3-b790-fccac7015ebf", "date": "Wed, 30 Apr 2025 06:43:59 GMT"} })
53082025-04-30T06:43:59.059ZDEBGtest_utilization (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 272.202205ms iteration = 2 status = {}
53092025-04-30T06:43:59.059ZTRACtest_utilization (SagaExecutor): loading roles actor = Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. } actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true resource_id = 001de000-5110-4000-8000-000000000000 resource_type = Silo saga_id = bc1ac890-e9e2-44d9-931e-3abe24cf75f5 saga_name = instance-start saga_node = EnsureRegistered
53102025-04-30T06:43:59.059ZTRACtest_utilization (SagaExecutor): authorize begin action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true resource = Database saga_id = bc1ac890-e9e2-44d9-931e-3abe24cf75f5 saga_name = instance-start saga_node = EnsureRegistered
53112025-04-30T06:43:59.059ZDEBGtest_utilization (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true roles = RoleSet { roles: {} } saga_id = bc1ac890-e9e2-44d9-931e-3abe24cf75f5 saga_name = instance-start saga_node = EnsureRegistered
53122025-04-30T06:43:59.061ZDEBGtest_utilization (SagaExecutor): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true resource = Database result = Ok(()) saga_id = bc1ac890-e9e2-44d9-931e-3abe24cf75f5 saga_name = instance-start saga_node = EnsureRegistered
53132025-04-30T06:43:59.065ZTRACtest_utilization (SagaExecutor): loading roles actor = Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. } actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet saga_id = bc1ac890-e9e2-44d9-931e-3abe24cf75f5 saga_name = instance-start saga_node = EnsureRegistered
53142025-04-30T06:43:59.065ZTRACtest_utilization (SagaExecutor): authorize begin action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true resource = Database saga_id = bc1ac890-e9e2-44d9-931e-3abe24cf75f5 saga_name = instance-start saga_node = EnsureRegistered
53152025-04-30T06:43:59.065ZDEBGtest_utilization (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true roles = RoleSet { roles: {} } saga_id = bc1ac890-e9e2-44d9-931e-3abe24cf75f5 saga_name = instance-start saga_node = EnsureRegistered
53162025-04-30T06:43:59.065ZDEBGtest_utilization (SagaExecutor): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true resource = Database result = Ok(()) saga_id = bc1ac890-e9e2-44d9-931e-3abe24cf75f5 saga_name = instance-start saga_node = EnsureRegistered
53172025-04-30T06:43:59.069ZDEBGtest_utilization (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin"), (Silo, 001de000-5110-4000-8000-000000000000, "admin")} } saga_id = bc1ac890-e9e2-44d9-931e-3abe24cf75f5 saga_name = instance-start saga_node = EnsureRegistered
53182025-04-30T06:43:59.076ZDEBGtest_utilization (SagaExecutor): authorize result action = ListChildren actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true resource = Instance { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000000, lookup_type: ById(001de000-5110-4000-8000-000000000000) }, key: 473b39f9-8669-45b3-b3d7-793f7e10fc31, lookup_type: ById(473b39f9-8669-45b3-b3d7-793f7e10fc31) }, key: d7bd2deb-ee27-4661-9e9a-524d734cd6ec, lookup_type: ById(d7bd2deb-ee27-4661-9e9a-524d734cd6ec) } result = Ok(()) saga_id = bc1ac890-e9e2-44d9-931e-3abe24cf75f5 saga_name = instance-start saga_node = EnsureRegistered
53192025-04-30T06:43:59.076ZTRACtest_utilization (SagaExecutor): authorize begin action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true resource = Database saga_id = bc1ac890-e9e2-44d9-931e-3abe24cf75f5 saga_name = instance-start saga_node = EnsureRegistered
53202025-04-30T06:43:59.076ZDEBGtest_utilization (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true roles = RoleSet { roles: {} } saga_id = bc1ac890-e9e2-44d9-931e-3abe24cf75f5 saga_name = instance-start saga_node = EnsureRegistered
53212025-04-30T06:43:59.077ZDEBGtest_utilization (SagaExecutor): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true resource = Database result = Ok(()) saga_id = bc1ac890-e9e2-44d9-931e-3abe24cf75f5 saga_name = instance-start saga_node = EnsureRegistered
53222025-04-30T06:43:59.105ZTRACtest_utilization (SagaExecutor): authorize begin action = ListChildren actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true resource = Instance { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000000, lookup_type: ById(001de000-5110-4000-8000-000000000000) }, key: 473b39f9-8669-45b3-b3d7-793f7e10fc31, lookup_type: ById(473b39f9-8669-45b3-b3d7-793f7e10fc31) }, key: d7bd2deb-ee27-4661-9e9a-524d734cd6ec, lookup_type: ById(d7bd2deb-ee27-4661-9e9a-524d734cd6ec) } saga_id = bc1ac890-e9e2-44d9-931e-3abe24cf75f5 saga_name = instance-start saga_node = EnsureRegistered
53232025-04-30T06:43:59.105ZTRACtest_utilization (SagaExecutor): loading roles actor = Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. } actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true resource_id = 473b39f9-8669-45b3-b3d7-793f7e10fc31 resource_type = Project saga_id = bc1ac890-e9e2-44d9-931e-3abe24cf75f5 saga_name = instance-start saga_node = EnsureRegistered
53242025-04-30T06:43:59.105ZTRACtest_utilization (SagaExecutor): authorize begin action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true resource = Database saga_id = bc1ac890-e9e2-44d9-931e-3abe24cf75f5 saga_name = instance-start saga_node = EnsureRegistered
53252025-04-30T06:43:59.105ZDEBGtest_utilization (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true roles = RoleSet { roles: {} } saga_id = bc1ac890-e9e2-44d9-931e-3abe24cf75f5 saga_name = instance-start saga_node = EnsureRegistered
53262025-04-30T06:43:59.106ZDEBGtest_utilization (SagaExecutor): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true resource = Database result = Ok(()) saga_id = bc1ac890-e9e2-44d9-931e-3abe24cf75f5 saga_name = instance-start saga_node = EnsureRegistered
53272025-04-30T06:43:59.133ZTRACtest_utilization (SagaExecutor): loading roles actor = Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. } actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true resource_id = 001de000-5110-4000-8000-000000000000 resource_type = Silo saga_id = bc1ac890-e9e2-44d9-931e-3abe24cf75f5 saga_name = instance-start saga_node = EnsureRegistered
53282025-04-30T06:43:59.140ZTRACtest_utilization (SagaExecutor): authorize begin action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true resource = Database saga_id = bc1ac890-e9e2-44d9-931e-3abe24cf75f5 saga_name = instance-start saga_node = EnsureRegistered
53292025-04-30T06:43:59.141ZDEBGtest_utilization (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true roles = RoleSet { roles: {} } saga_id = bc1ac890-e9e2-44d9-931e-3abe24cf75f5 saga_name = instance-start saga_node = EnsureRegistered
53302025-04-30T06:43:59.141ZDEBGtest_utilization (SagaExecutor): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true resource = Database result = Ok(()) saga_id = bc1ac890-e9e2-44d9-931e-3abe24cf75f5 saga_name = instance-start saga_node = EnsureRegistered
53312025-04-30T06:43:59.148ZTRACtest_utilization (SagaExecutor): loading roles actor = Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. } actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet saga_id = bc1ac890-e9e2-44d9-931e-3abe24cf75f5 saga_name = instance-start saga_node = EnsureRegistered
53322025-04-30T06:43:59.148ZTRACtest_utilization (SagaExecutor): authorize begin action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true resource = Database saga_id = bc1ac890-e9e2-44d9-931e-3abe24cf75f5 saga_name = instance-start saga_node = EnsureRegistered
53332025-04-30T06:43:59.148ZDEBGtest_utilization (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true roles = RoleSet { roles: {} } saga_id = bc1ac890-e9e2-44d9-931e-3abe24cf75f5 saga_name = instance-start saga_node = EnsureRegistered
53342025-04-30T06:43:59.149ZDEBGtest_utilization (SagaExecutor): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true resource = Database result = Ok(()) saga_id = bc1ac890-e9e2-44d9-931e-3abe24cf75f5 saga_name = instance-start saga_node = EnsureRegistered
53352025-04-30T06:43:59.162ZDEBGtest_utilization (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin"), (Silo, 001de000-5110-4000-8000-000000000000, "admin")} } saga_id = bc1ac890-e9e2-44d9-931e-3abe24cf75f5 saga_name = instance-start saga_node = EnsureRegistered
53362025-04-30T06:43:59.170ZDEBGtest_utilization (SagaExecutor): authorize result action = ListChildren actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true resource = Instance { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000000, lookup_type: ById(001de000-5110-4000-8000-000000000000) }, key: 473b39f9-8669-45b3-b3d7-793f7e10fc31, lookup_type: ById(473b39f9-8669-45b3-b3d7-793f7e10fc31) }, key: d7bd2deb-ee27-4661-9e9a-524d734cd6ec, lookup_type: ById(d7bd2deb-ee27-4661-9e9a-524d734cd6ec) } result = Ok(()) saga_id = bc1ac890-e9e2-44d9-931e-3abe24cf75f5 saga_name = instance-start saga_node = EnsureRegistered
53372025-04-30T06:43:59.170ZTRACtest_utilization (SagaExecutor): authorize begin action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true resource = Database saga_id = bc1ac890-e9e2-44d9-931e-3abe24cf75f5 saga_name = instance-start saga_node = EnsureRegistered
53382025-04-30T06:43:59.170ZDEBGtest_utilization (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true roles = RoleSet { roles: {} } saga_id = bc1ac890-e9e2-44d9-931e-3abe24cf75f5 saga_name = instance-start saga_node = EnsureRegistered
53392025-04-30T06:43:59.170ZDEBGtest_utilization (SagaExecutor): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true resource = Database result = Ok(()) saga_id = bc1ac890-e9e2-44d9-931e-3abe24cf75f5 saga_name = instance-start saga_node = EnsureRegistered
53402025-04-30T06:43:59.178ZTRACtest_utilization (SagaExecutor): authorize begin action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true resource = Database saga_id = bc1ac890-e9e2-44d9-931e-3abe24cf75f5 saga_name = instance-start saga_node = EnsureRegistered
53412025-04-30T06:43:59.178ZDEBGtest_utilization (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true roles = RoleSet { roles: {} } saga_id = bc1ac890-e9e2-44d9-931e-3abe24cf75f5 saga_name = instance-start saga_node = EnsureRegistered
53422025-04-30T06:43:59.179ZDEBGtest_utilization (SagaExecutor): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true resource = Database result = Ok(()) saga_id = bc1ac890-e9e2-44d9-931e-3abe24cf75f5 saga_name = instance-start saga_node = EnsureRegistered
53432025-04-30T06:43:59.185ZTRACtest_utilization (SagaExecutor): authorize begin action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true resource = Database saga_id = bc1ac890-e9e2-44d9-931e-3abe24cf75f5 saga_name = instance-start saga_node = EnsureRegistered
53442025-04-30T06:43:59.186ZDEBGtest_utilization (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true roles = RoleSet { roles: {} } saga_id = bc1ac890-e9e2-44d9-931e-3abe24cf75f5 saga_name = instance-start saga_node = EnsureRegistered
53452025-04-30T06:43:59.189ZDEBGtest_utilization (SagaExecutor): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true resource = Database result = Ok(()) saga_id = bc1ac890-e9e2-44d9-931e-3abe24cf75f5 saga_name = instance-start saga_node = EnsureRegistered
53462025-04-30T06:43:59.196ZTRACtest_utilization (SagaExecutor): authorize begin action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true resource = Database saga_id = bc1ac890-e9e2-44d9-931e-3abe24cf75f5 saga_name = instance-start saga_node = EnsureRegistered
53472025-04-30T06:43:59.196ZDEBGtest_utilization (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true roles = RoleSet { roles: {} } saga_id = bc1ac890-e9e2-44d9-931e-3abe24cf75f5 saga_name = instance-start saga_node = EnsureRegistered
53482025-04-30T06:43:59.197ZDEBGtest_utilization (SagaExecutor): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true resource = Database result = Ok(()) saga_id = bc1ac890-e9e2-44d9-931e-3abe24cf75f5 saga_name = instance-start saga_node = EnsureRegistered
53492025-04-30T06:43:59.200ZTRACtest_utilization (SagaExecutor): authorize begin action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true resource = Database saga_id = bc1ac890-e9e2-44d9-931e-3abe24cf75f5 saga_name = instance-start saga_node = EnsureRegistered
53502025-04-30T06:43:59.200ZDEBGtest_utilization (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true roles = RoleSet { roles: {} } saga_id = bc1ac890-e9e2-44d9-931e-3abe24cf75f5 saga_name = instance-start saga_node = EnsureRegistered
53512025-04-30T06:43:59.201ZDEBGtest_utilization (SagaExecutor): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true resource = Database result = Ok(()) saga_id = bc1ac890-e9e2-44d9-931e-3abe24cf75f5 saga_name = instance-start saga_node = EnsureRegistered
53522025-04-30T06:43:59.206ZTRACtest_utilization (SagaExecutor): authorize begin action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true resource = Database saga_id = bc1ac890-e9e2-44d9-931e-3abe24cf75f5 saga_name = instance-start saga_node = EnsureRegistered
53532025-04-30T06:43:59.206ZDEBGtest_utilization (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true roles = RoleSet { roles: {} } saga_id = bc1ac890-e9e2-44d9-931e-3abe24cf75f5 saga_name = instance-start saga_node = EnsureRegistered
53542025-04-30T06:43:59.207ZDEBGtest_utilization (SagaExecutor): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true resource = Database result = Ok(()) saga_id = bc1ac890-e9e2-44d9-931e-3abe24cf75f5 saga_name = instance-start saga_node = EnsureRegistered
53552025-04-30T06:43:59.211ZTRACtest_utilization (SagaExecutor): authorize begin action = Read actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000000, lookup_type: ById(001de000-5110-4000-8000-000000000000) }, key: 473b39f9-8669-45b3-b3d7-793f7e10fc31, lookup_type: ById(473b39f9-8669-45b3-b3d7-793f7e10fc31) }, key: cda41110-774f-439b-b99b-f715ebb74e33, lookup_type: ById(cda41110-774f-439b-b99b-f715ebb74e33) } saga_id = bc1ac890-e9e2-44d9-931e-3abe24cf75f5 saga_name = instance-start saga_node = EnsureRegistered
53562025-04-30T06:43:59.212ZTRACtest_utilization (SagaExecutor): loading roles actor = Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. } actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true resource_id = 473b39f9-8669-45b3-b3d7-793f7e10fc31 resource_type = Project saga_id = bc1ac890-e9e2-44d9-931e-3abe24cf75f5 saga_name = instance-start saga_node = EnsureRegistered
53572025-04-30T06:43:59.212ZTRACtest_utilization (SagaExecutor): authorize begin action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true resource = Database saga_id = bc1ac890-e9e2-44d9-931e-3abe24cf75f5 saga_name = instance-start saga_node = EnsureRegistered
53582025-04-30T06:43:59.212ZDEBGtest_utilization (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true roles = RoleSet { roles: {} } saga_id = bc1ac890-e9e2-44d9-931e-3abe24cf75f5 saga_name = instance-start saga_node = EnsureRegistered
53592025-04-30T06:43:59.212ZDEBGtest_utilization (SagaExecutor): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true resource = Database result = Ok(()) saga_id = bc1ac890-e9e2-44d9-931e-3abe24cf75f5 saga_name = instance-start saga_node = EnsureRegistered
53602025-04-30T06:43:59.216ZTRACtest_utilization (SagaExecutor): loading roles actor = Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. } actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true resource_id = 001de000-5110-4000-8000-000000000000 resource_type = Silo saga_id = bc1ac890-e9e2-44d9-931e-3abe24cf75f5 saga_name = instance-start saga_node = EnsureRegistered
53612025-04-30T06:43:59.216ZTRACtest_utilization (SagaExecutor): authorize begin action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true resource = Database saga_id = bc1ac890-e9e2-44d9-931e-3abe24cf75f5 saga_name = instance-start saga_node = EnsureRegistered
53622025-04-30T06:43:59.216ZDEBGtest_utilization (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true roles = RoleSet { roles: {} } saga_id = bc1ac890-e9e2-44d9-931e-3abe24cf75f5 saga_name = instance-start saga_node = EnsureRegistered
53632025-04-30T06:43:59.217ZDEBGtest_utilization (SagaExecutor): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true resource = Database result = Ok(()) saga_id = bc1ac890-e9e2-44d9-931e-3abe24cf75f5 saga_name = instance-start saga_node = EnsureRegistered
53642025-04-30T06:43:59.225ZTRACtest_utilization (SagaExecutor): loading roles actor = Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. } actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet saga_id = bc1ac890-e9e2-44d9-931e-3abe24cf75f5 saga_name = instance-start saga_node = EnsureRegistered
53652025-04-30T06:43:59.227ZTRACtest_utilization (SagaExecutor): authorize begin action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true resource = Database saga_id = bc1ac890-e9e2-44d9-931e-3abe24cf75f5 saga_name = instance-start saga_node = EnsureRegistered
53662025-04-30T06:43:59.228ZDEBGtest_utilization (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true roles = RoleSet { roles: {} } saga_id = bc1ac890-e9e2-44d9-931e-3abe24cf75f5 saga_name = instance-start saga_node = EnsureRegistered
53672025-04-30T06:43:59.228ZDEBGtest_utilization (SagaExecutor): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true resource = Database result = Ok(()) saga_id = bc1ac890-e9e2-44d9-931e-3abe24cf75f5 saga_name = instance-start saga_node = EnsureRegistered
53682025-04-30T06:43:59.238ZDEBGtest_utilization (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin"), (Silo, 001de000-5110-4000-8000-000000000000, "admin")} } saga_id = bc1ac890-e9e2-44d9-931e-3abe24cf75f5 saga_name = instance-start saga_node = EnsureRegistered
53692025-04-30T06:43:59.254ZDEBGtest_utilization (SagaExecutor): authorize result action = Read actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000000, lookup_type: ById(001de000-5110-4000-8000-000000000000) }, key: 473b39f9-8669-45b3-b3d7-793f7e10fc31, lookup_type: ById(473b39f9-8669-45b3-b3d7-793f7e10fc31) }, key: cda41110-774f-439b-b99b-f715ebb74e33, lookup_type: ById(cda41110-774f-439b-b99b-f715ebb74e33) } result = Ok(()) saga_id = bc1ac890-e9e2-44d9-931e-3abe24cf75f5 saga_name = instance-start saga_node = EnsureRegistered
53702025-04-30T06:43:59.254ZTRACtest_utilization (SagaExecutor): authorize begin action = Read actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000000, lookup_type: ById(001de000-5110-4000-8000-000000000000) }, key: 473b39f9-8669-45b3-b3d7-793f7e10fc31, lookup_type: ById(473b39f9-8669-45b3-b3d7-793f7e10fc31) }, key: cda41110-774f-439b-b99b-f715ebb74e33, lookup_type: ById(cda41110-774f-439b-b99b-f715ebb74e33) } saga_id = bc1ac890-e9e2-44d9-931e-3abe24cf75f5 saga_name = instance-start saga_node = EnsureRegistered
53712025-04-30T06:43:59.254ZTRACtest_utilization (SagaExecutor): loading roles actor = Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. } actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true resource_id = 473b39f9-8669-45b3-b3d7-793f7e10fc31 resource_type = Project saga_id = bc1ac890-e9e2-44d9-931e-3abe24cf75f5 saga_name = instance-start saga_node = EnsureRegistered
53722025-04-30T06:43:59.254ZTRACtest_utilization (SagaExecutor): authorize begin action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true resource = Database saga_id = bc1ac890-e9e2-44d9-931e-3abe24cf75f5 saga_name = instance-start saga_node = EnsureRegistered
53732025-04-30T06:43:59.254ZDEBGtest_utilization (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true roles = RoleSet { roles: {} } saga_id = bc1ac890-e9e2-44d9-931e-3abe24cf75f5 saga_name = instance-start saga_node = EnsureRegistered
53742025-04-30T06:43:59.256ZDEBGtest_utilization (SagaExecutor): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true resource = Database result = Ok(()) saga_id = bc1ac890-e9e2-44d9-931e-3abe24cf75f5 saga_name = instance-start saga_node = EnsureRegistered
53752025-04-30T06:43:59.273ZTRACtest_utilization (SagaExecutor): loading roles actor = Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. } actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true resource_id = 001de000-5110-4000-8000-000000000000 resource_type = Silo saga_id = bc1ac890-e9e2-44d9-931e-3abe24cf75f5 saga_name = instance-start saga_node = EnsureRegistered
53762025-04-30T06:43:59.273ZTRACtest_utilization (SagaExecutor): authorize begin action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true resource = Database saga_id = bc1ac890-e9e2-44d9-931e-3abe24cf75f5 saga_name = instance-start saga_node = EnsureRegistered
53772025-04-30T06:43:59.273ZDEBGtest_utilization (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true roles = RoleSet { roles: {} } saga_id = bc1ac890-e9e2-44d9-931e-3abe24cf75f5 saga_name = instance-start saga_node = EnsureRegistered
53782025-04-30T06:43:59.274ZDEBGtest_utilization (SagaExecutor): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true resource = Database result = Ok(()) saga_id = bc1ac890-e9e2-44d9-931e-3abe24cf75f5 saga_name = instance-start saga_node = EnsureRegistered
53792025-04-30T06:43:59.284ZTRACtest_utilization (SagaExecutor): loading roles actor = Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. } actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet saga_id = bc1ac890-e9e2-44d9-931e-3abe24cf75f5 saga_name = instance-start saga_node = EnsureRegistered
53802025-04-30T06:43:59.284ZTRACtest_utilization (SagaExecutor): authorize begin action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true resource = Database saga_id = bc1ac890-e9e2-44d9-931e-3abe24cf75f5 saga_name = instance-start saga_node = EnsureRegistered
53812025-04-30T06:43:59.284ZDEBGtest_utilization (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true roles = RoleSet { roles: {} } saga_id = bc1ac890-e9e2-44d9-931e-3abe24cf75f5 saga_name = instance-start saga_node = EnsureRegistered
53822025-04-30T06:43:59.285ZDEBGtest_utilization (SagaExecutor): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true resource = Database result = Ok(()) saga_id = bc1ac890-e9e2-44d9-931e-3abe24cf75f5 saga_name = instance-start saga_node = EnsureRegistered
53832025-04-30T06:43:59.293ZDEBGtest_utilization (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin"), (Silo, 001de000-5110-4000-8000-000000000000, "admin")} } saga_id = bc1ac890-e9e2-44d9-931e-3abe24cf75f5 saga_name = instance-start saga_node = EnsureRegistered
53842025-04-30T06:43:59.303ZDEBGtest_utilization (SagaExecutor): authorize result action = Read actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000000, lookup_type: ById(001de000-5110-4000-8000-000000000000) }, key: 473b39f9-8669-45b3-b3d7-793f7e10fc31, lookup_type: ById(473b39f9-8669-45b3-b3d7-793f7e10fc31) }, key: cda41110-774f-439b-b99b-f715ebb74e33, lookup_type: ById(cda41110-774f-439b-b99b-f715ebb74e33) } result = Ok(()) saga_id = bc1ac890-e9e2-44d9-931e-3abe24cf75f5 saga_name = instance-start saga_node = EnsureRegistered
53852025-04-30T06:43:59.303ZTRACtest_utilization (SagaExecutor): authorize begin action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true resource = Database saga_id = bc1ac890-e9e2-44d9-931e-3abe24cf75f5 saga_name = instance-start saga_node = EnsureRegistered
53862025-04-30T06:43:59.303ZDEBGtest_utilization (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true roles = RoleSet { roles: {} } saga_id = bc1ac890-e9e2-44d9-931e-3abe24cf75f5 saga_name = instance-start saga_node = EnsureRegistered
53872025-04-30T06:43:59.304ZDEBGtest_utilization (SagaExecutor): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true resource = Database result = Ok(()) saga_id = bc1ac890-e9e2-44d9-931e-3abe24cf75f5 saga_name = instance-start saga_node = EnsureRegistered
53882025-04-30T06:43:59.327ZTRACtest_utilization (SagaExecutor): authorize begin action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true resource = Database saga_id = bc1ac890-e9e2-44d9-931e-3abe24cf75f5 saga_name = instance-start saga_node = EnsureRegistered
53892025-04-30T06:43:59.328ZDEBGtest_utilization (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true roles = RoleSet { roles: {} } saga_id = bc1ac890-e9e2-44d9-931e-3abe24cf75f5 saga_name = instance-start saga_node = EnsureRegistered
53902025-04-30T06:43:59.329ZDEBGtest_utilization (SagaExecutor): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true resource = Database result = Ok(()) saga_id = bc1ac890-e9e2-44d9-931e-3abe24cf75f5 saga_name = instance-start saga_node = EnsureRegistered
53912025-04-30T06:43:59.333ZTRACtest_utilization (SagaExecutor): authorize begin action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true resource = Database saga_id = bc1ac890-e9e2-44d9-931e-3abe24cf75f5 saga_name = instance-start saga_node = EnsureRegistered
53922025-04-30T06:43:59.333ZDEBGtest_utilization (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true roles = RoleSet { roles: {} } saga_id = bc1ac890-e9e2-44d9-931e-3abe24cf75f5 saga_name = instance-start saga_node = EnsureRegistered
53932025-04-30T06:43:59.334ZDEBGtest_utilization (SagaExecutor): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true resource = Database result = Ok(()) saga_id = bc1ac890-e9e2-44d9-931e-3abe24cf75f5 saga_name = instance-start saga_node = EnsureRegistered
53942025-04-30T06:43:59.336ZTRACtest_utilization (SagaExecutor): authorize begin action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true resource = Database saga_id = bc1ac890-e9e2-44d9-931e-3abe24cf75f5 saga_name = instance-start saga_node = EnsureRegistered
53952025-04-30T06:43:59.336ZDEBGtest_utilization (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true roles = RoleSet { roles: {} } saga_id = bc1ac890-e9e2-44d9-931e-3abe24cf75f5 saga_name = instance-start saga_node = EnsureRegistered
53962025-04-30T06:43:59.337ZDEBGtest_utilization (SagaExecutor): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true resource = Database result = Ok(()) saga_id = bc1ac890-e9e2-44d9-931e-3abe24cf75f5 saga_name = instance-start saga_node = EnsureRegistered
53972025-04-30T06:43:59.342ZTRACtest_utilization (SagaExecutor): authorize begin action = ListChildren actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000000, lookup_type: ById(001de000-5110-4000-8000-000000000000) }, key: 473b39f9-8669-45b3-b3d7-793f7e10fc31, lookup_type: ById(473b39f9-8669-45b3-b3d7-793f7e10fc31) }, key: cda41110-774f-439b-b99b-f715ebb74e33, lookup_type: ByName("default") } saga_id = bc1ac890-e9e2-44d9-931e-3abe24cf75f5 saga_name = instance-start saga_node = EnsureRegistered
53982025-04-30T06:43:59.343ZTRACtest_utilization (SagaExecutor): loading roles actor = Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. } actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true resource_id = 473b39f9-8669-45b3-b3d7-793f7e10fc31 resource_type = Project saga_id = bc1ac890-e9e2-44d9-931e-3abe24cf75f5 saga_name = instance-start saga_node = EnsureRegistered
53992025-04-30T06:43:59.343ZTRACtest_utilization (SagaExecutor): authorize begin action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true resource = Database saga_id = bc1ac890-e9e2-44d9-931e-3abe24cf75f5 saga_name = instance-start saga_node = EnsureRegistered
54002025-04-30T06:43:59.343ZDEBGtest_utilization (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true roles = RoleSet { roles: {} } saga_id = bc1ac890-e9e2-44d9-931e-3abe24cf75f5 saga_name = instance-start saga_node = EnsureRegistered
54012025-04-30T06:43:59.344ZDEBGtest_utilization (SagaExecutor): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true resource = Database result = Ok(()) saga_id = bc1ac890-e9e2-44d9-931e-3abe24cf75f5 saga_name = instance-start saga_node = EnsureRegistered
54022025-04-30T06:43:59.348ZTRACtest_utilization (SagaExecutor): loading roles actor = Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. } actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true resource_id = 001de000-5110-4000-8000-000000000000 resource_type = Silo saga_id = bc1ac890-e9e2-44d9-931e-3abe24cf75f5 saga_name = instance-start saga_node = EnsureRegistered
54032025-04-30T06:43:59.348ZTRACtest_utilization (SagaExecutor): authorize begin action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true resource = Database saga_id = bc1ac890-e9e2-44d9-931e-3abe24cf75f5 saga_name = instance-start saga_node = EnsureRegistered
54042025-04-30T06:43:59.348ZDEBGtest_utilization (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true roles = RoleSet { roles: {} } saga_id = bc1ac890-e9e2-44d9-931e-3abe24cf75f5 saga_name = instance-start saga_node = EnsureRegistered
54052025-04-30T06:43:59.348ZDEBGtest_utilization (SagaExecutor): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true resource = Database result = Ok(()) saga_id = bc1ac890-e9e2-44d9-931e-3abe24cf75f5 saga_name = instance-start saga_node = EnsureRegistered
54062025-04-30T06:43:59.365ZDEBGtest_utilization (results-sink): inserting 1 samples into database collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
54072025-04-30T06:43:59.365ZDEBGtest_utilization (results-sink-cluster): inserting 1 samples into database collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
54082025-04-30T06:43:59.365ZTRACtest_utilization (SagaExecutor): loading roles actor = Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. } actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet saga_id = bc1ac890-e9e2-44d9-931e-3abe24cf75f5 saga_name = instance-start saga_node = EnsureRegistered
54092025-04-30T06:43:59.365ZTRACtest_utilization (SagaExecutor): authorize begin action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true resource = Database saga_id = bc1ac890-e9e2-44d9-931e-3abe24cf75f5 saga_name = instance-start saga_node = EnsureRegistered
54102025-04-30T06:43:59.365ZDEBGtest_utilization (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true roles = RoleSet { roles: {} } saga_id = bc1ac890-e9e2-44d9-931e-3abe24cf75f5 saga_name = instance-start saga_node = EnsureRegistered
54112025-04-30T06:43:59.366ZDEBGtest_utilization (SagaExecutor): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true resource = Database result = Ok(()) saga_id = bc1ac890-e9e2-44d9-931e-3abe24cf75f5 saga_name = instance-start saga_node = EnsureRegistered
54122025-04-30T06:43:59.366ZDEBGtest_utilization (clickhouse-client): unrolling 1 total samples collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 4d1485f0-110e-4161-9a33-c4610d68be5d
54132025-04-30T06:43:59.368ZTRACtest_utilization (clickhouse-client): inserting data collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 4d1485f0-110e-4161-9a33-c4610d68be5d n_columns = 4 n_rows = 2 sql = INSERT INTO oximeter.fields_string FORMAT Native
54142025-04-30T06:43:59.372ZDEBGtest_utilization (clickhouse-client): inserted rows into table collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 4d1485f0-110e-4161-9a33-c4610d68be5d n_rows = 2 table_name = fields_string
54152025-04-30T06:43:59.372ZTRACtest_utilization (clickhouse-client): inserting data collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 4d1485f0-110e-4161-9a33-c4610d68be5d n_columns = 4 n_rows = 1 sql = INSERT INTO oximeter.measurements_i64 FORMAT Native
54162025-04-30T06:43:59.378ZDEBGtest_utilization (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin"), (Silo, 001de000-5110-4000-8000-000000000000, "admin")} } saga_id = bc1ac890-e9e2-44d9-931e-3abe24cf75f5 saga_name = instance-start saga_node = EnsureRegistered
54172025-04-30T06:43:59.385ZDEBGtest_utilization (SagaExecutor): authorize result action = ListChildren actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000000, lookup_type: ById(001de000-5110-4000-8000-000000000000) }, key: 473b39f9-8669-45b3-b3d7-793f7e10fc31, lookup_type: ById(473b39f9-8669-45b3-b3d7-793f7e10fc31) }, key: cda41110-774f-439b-b99b-f715ebb74e33, lookup_type: ByName("default") } result = Ok(()) saga_id = bc1ac890-e9e2-44d9-931e-3abe24cf75f5 saga_name = instance-start saga_node = EnsureRegistered
54182025-04-30T06:43:59.385ZTRACtest_utilization (SagaExecutor): authorize begin action = ListChildren actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000000, lookup_type: ById(001de000-5110-4000-8000-000000000000) }, key: 473b39f9-8669-45b3-b3d7-793f7e10fc31, lookup_type: ById(473b39f9-8669-45b3-b3d7-793f7e10fc31) }, key: cda41110-774f-439b-b99b-f715ebb74e33, lookup_type: ByName("default") } saga_id = bc1ac890-e9e2-44d9-931e-3abe24cf75f5 saga_name = instance-start saga_node = EnsureRegistered
54192025-04-30T06:43:59.385ZTRACtest_utilization (SagaExecutor): loading roles actor = Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. } actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true resource_id = 473b39f9-8669-45b3-b3d7-793f7e10fc31 resource_type = Project saga_id = bc1ac890-e9e2-44d9-931e-3abe24cf75f5 saga_name = instance-start saga_node = EnsureRegistered
54202025-04-30T06:43:59.385ZTRACtest_utilization (SagaExecutor): authorize begin action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true resource = Database saga_id = bc1ac890-e9e2-44d9-931e-3abe24cf75f5 saga_name = instance-start saga_node = EnsureRegistered
54212025-04-30T06:43:59.385ZDEBGtest_utilization (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true roles = RoleSet { roles: {} } saga_id = bc1ac890-e9e2-44d9-931e-3abe24cf75f5 saga_name = instance-start saga_node = EnsureRegistered
54222025-04-30T06:43:59.386ZDEBGtest_utilization (SagaExecutor): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true resource = Database result = Ok(()) saga_id = bc1ac890-e9e2-44d9-931e-3abe24cf75f5 saga_name = instance-start saga_node = EnsureRegistered
54232025-04-30T06:43:59.386ZDEBGtest_utilization (clickhouse-client): inserted rows into table collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 4d1485f0-110e-4161-9a33-c4610d68be5d n_rows = 1 table_name = measurements_i64
54242025-04-30T06:43:59.386ZTRACtest_utilization (results-sink): successfully inserted samples collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
54252025-04-30T06:43:59.389ZTRACtest_utilization (SagaExecutor): loading roles actor = Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. } actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true resource_id = 001de000-5110-4000-8000-000000000000 resource_type = Silo saga_id = bc1ac890-e9e2-44d9-931e-3abe24cf75f5 saga_name = instance-start saga_node = EnsureRegistered
54262025-04-30T06:43:59.389ZTRACtest_utilization (SagaExecutor): authorize begin action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true resource = Database saga_id = bc1ac890-e9e2-44d9-931e-3abe24cf75f5 saga_name = instance-start saga_node = EnsureRegistered
54272025-04-30T06:43:59.389ZDEBGtest_utilization (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true roles = RoleSet { roles: {} } saga_id = bc1ac890-e9e2-44d9-931e-3abe24cf75f5 saga_name = instance-start saga_node = EnsureRegistered
54282025-04-30T06:43:59.390ZDEBGtest_utilization (SagaExecutor): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true resource = Database result = Ok(()) saga_id = bc1ac890-e9e2-44d9-931e-3abe24cf75f5 saga_name = instance-start saga_node = EnsureRegistered
54292025-04-30T06:43:59.401ZTRACtest_utilization (SagaExecutor): loading roles actor = Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. } actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet saga_id = bc1ac890-e9e2-44d9-931e-3abe24cf75f5 saga_name = instance-start saga_node = EnsureRegistered
54302025-04-30T06:43:59.401ZTRACtest_utilization (SagaExecutor): authorize begin action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true resource = Database saga_id = bc1ac890-e9e2-44d9-931e-3abe24cf75f5 saga_name = instance-start saga_node = EnsureRegistered
54312025-04-30T06:43:59.401ZDEBGtest_utilization (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true roles = RoleSet { roles: {} } saga_id = bc1ac890-e9e2-44d9-931e-3abe24cf75f5 saga_name = instance-start saga_node = EnsureRegistered
54322025-04-30T06:43:59.401ZDEBGtest_utilization (SagaExecutor): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true resource = Database result = Ok(()) saga_id = bc1ac890-e9e2-44d9-931e-3abe24cf75f5 saga_name = instance-start saga_node = EnsureRegistered
54332025-04-30T06:43:59.413ZDEBGtest_utilization (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin"), (Silo, 001de000-5110-4000-8000-000000000000, "admin")} } saga_id = bc1ac890-e9e2-44d9-931e-3abe24cf75f5 saga_name = instance-start saga_node = EnsureRegistered
54342025-04-30T06:43:59.428ZDEBGtest_utilization (SagaExecutor): authorize result action = ListChildren actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000000, lookup_type: ById(001de000-5110-4000-8000-000000000000) }, key: 473b39f9-8669-45b3-b3d7-793f7e10fc31, lookup_type: ById(473b39f9-8669-45b3-b3d7-793f7e10fc31) }, key: cda41110-774f-439b-b99b-f715ebb74e33, lookup_type: ByName("default") } result = Ok(()) saga_id = bc1ac890-e9e2-44d9-931e-3abe24cf75f5 saga_name = instance-start saga_node = EnsureRegistered
54352025-04-30T06:43:59.428ZTRACtest_utilization (SagaExecutor): authorize begin action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true resource = Database saga_id = bc1ac890-e9e2-44d9-931e-3abe24cf75f5 saga_name = instance-start saga_node = EnsureRegistered
54362025-04-30T06:43:59.429ZDEBGtest_utilization (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true roles = RoleSet { roles: {} } saga_id = bc1ac890-e9e2-44d9-931e-3abe24cf75f5 saga_name = instance-start saga_node = EnsureRegistered
54372025-04-30T06:43:59.429ZDEBGtest_utilization (SagaExecutor): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true resource = Database result = Ok(()) saga_id = bc1ac890-e9e2-44d9-931e-3abe24cf75f5 saga_name = instance-start saga_node = EnsureRegistered
54382025-04-30T06:43:59.465ZDEBGtest_utilization (nexus): resolved names for firewall rules instance_interfaces = {} subnet_interfaces = {} subnet_networks = {} vpc_interfaces = {Name("default"): [NetworkInterface { id: 64c48731-6cc1-46ff-99f2-86f6c481d0c6, kind: Instance { id: d7bd2deb-ee27-4661-9e9a-524d734cd6ec }, name: Name("net0"), ip: 172.30.0.5, mac: MacAddr(MacAddr6([168, 64, 37, 240, 0, 0])), subnet: V4(Ipv4Net { addr: 172.30.0.0, width: 22 }), vni: Vni(11911937), primary: true, slot: 0, transit_ips: [] }]}
54392025-04-30T06:43:59.465ZDEBGtest_utilization (nexus): resolved firewall rules for sled agents sled_agent_rules = [ResolvedVpcFirewallRule { action: Allow, direction: Inbound, filter_hosts: None, filter_ports: None, filter_protocols: Some([Icmp]), priority: 65534, status: Enabled, targets: [NetworkInterface { id: 64c48731-6cc1-46ff-99f2-86f6c481d0c6, kind: Instance { id: d7bd2deb-ee27-4661-9e9a-524d734cd6ec }, name: Name("net0"), ip: 172.30.0.5, mac: MacAddr(MacAddr6([168, 64, 37, 240, 0, 0])), subnet: V4(Ipv4Net { addr: 172.30.0.0, width: 22 }), vni: Vni(11911937), primary: true, slot: 0, transit_ips: [] }] }, ResolvedVpcFirewallRule { action: Allow, direction: Inbound, filter_hosts: Some([Vpc(Vni(11911937))]), filter_ports: None, filter_protocols: None, priority: 65534, status: Enabled, targets: [NetworkInterface { id: 64c48731-6cc1-46ff-99f2-86f6c481d0c6, kind: Instance { id: d7bd2deb-ee27-4661-9e9a-524d734cd6ec }, name: Name("net0"), ip: 172.30.0.5, mac: MacAddr(MacAddr6([168, 64, 37, 240, 0, 0])), subnet: V4(Ipv4Net { addr: 172.30.0.0, width: 22 }), vni: Vni(11911937), primary: true, slot: 0, transit_ips: [] }] }, ResolvedVpcFirewallRule { action: Allow, direction: Inbound, filter_hosts: None, filter_ports: Some([L4PortRange("22")]), filter_protocols: Some([Tcp]), priority: 65534, status: Enabled, targets: [NetworkInterface { id: 64c48731-6cc1-46ff-99f2-86f6c481d0c6, kind: Instance { id: d7bd2deb-ee27-4661-9e9a-524d734cd6ec }, name: Name("net0"), ip: 172.30.0.5, mac: MacAddr(MacAddr6([168, 64, 37, 240, 0, 0])), subnet: V4(Ipv4Net { addr: 172.30.0.0, width: 22 }), vni: Vni(11911937), primary: true, slot: 0, transit_ips: [] }] }]
54402025-04-30T06:43:59.465ZTRACtest_utilization (SagaExecutor): authorize begin action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true resource = Database saga_id = bc1ac890-e9e2-44d9-931e-3abe24cf75f5 saga_name = instance-start saga_node = EnsureRegistered
54412025-04-30T06:43:59.465ZDEBGtest_utilization (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true roles = RoleSet { roles: {} } saga_id = bc1ac890-e9e2-44d9-931e-3abe24cf75f5 saga_name = instance-start saga_node = EnsureRegistered
54422025-04-30T06:43:59.466ZDEBGtest_utilization (SagaExecutor): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true resource = Database result = Ok(()) saga_id = bc1ac890-e9e2-44d9-931e-3abe24cf75f5 saga_name = instance-start saga_node = EnsureRegistered
54432025-04-30T06:43:59.467ZWARNtest_utilization (results-sink-cluster): failed to insert some results into metric DB collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 error = Failed to check out connection to database: No backends found for this service file = oximeter/collector/src/results_sink.rs:92
54442025-04-30T06:43:59.476ZTRACtest_utilization (InstanceAllocator): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000004ead, .. }) resource = Database
54452025-04-30T06:43:59.476ZDEBGtest_utilization (InstanceAllocator): roles roles = RoleSet { roles: {} }
54462025-04-30T06:43:59.477ZDEBGtest_utilization (InstanceAllocator): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000004ead, .. }) resource = Database result = Ok(())
54472025-04-30T06:43:59.480ZTRACtest_utilization (InstanceAllocator): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000004ead, .. }) resource = Sled { parent: Fleet, key: b6d65341-167c-41df-9b5c-41cded99c229, lookup_type: ById(b6d65341-167c-41df-9b5c-41cded99c229) }
54482025-04-30T06:43:59.480ZTRACtest_utilization (InstanceAllocator): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000004ead, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
54492025-04-30T06:43:59.480ZTRACtest_utilization (InstanceAllocator): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000004ead, .. }) resource = Database
54502025-04-30T06:43:59.480ZDEBGtest_utilization (InstanceAllocator): roles roles = RoleSet { roles: {} }
54512025-04-30T06:43:59.481ZDEBGtest_utilization (InstanceAllocator): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000004ead, .. }) resource = Database result = Ok(())
54522025-04-30T06:43:59.488ZDEBGtest_utilization (InstanceAllocator): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "viewer")} }
54532025-04-30T06:43:59.497ZDEBGtest_utilization (InstanceAllocator): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000004ead, .. }) resource = Sled { parent: Fleet, key: b6d65341-167c-41df-9b5c-41cded99c229, lookup_type: ById(b6d65341-167c-41df-9b5c-41cded99c229) } result = Ok(())
54542025-04-30T06:43:59.549ZDEBGtest_utilization (nexus): client request SledAgent = b6d65341-167c-41df-9b5c-41cded99c229 body = Some(Body) method = PUT uri = http://[::1]:39365/vmms/9e8dbed7-faa3-4634-886b-3a005dd8e99b
54552025-04-30T06:43:59.550ZINFOtest_utilization (dropshot): accepted connection local_addr = [::1]:39365 remote_addr = [::1]:46906 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
54562025-04-30T06:43:59.551ZTRACtest_utilization (dropshot): incoming request local_addr = [::1]:39365 method = PUT remote_addr = [::1]:46906 req_id = bed4da72-ac57-454c-af79-b12d16034ded sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /vmms/9e8dbed7-faa3-4634-886b-3a005dd8e99b
54572025-04-30T06:43:59.553ZINFOtest_utilization (SledAgent): created id = 9e8dbed7-faa3-4634-886b-3a005dd8e99b initial_state = SledVmmState { vmm_state: VmmRuntimeState { state: Starting, gen: Generation(2), time_updated: 2025-04-30T06:43:59.549025164Z }, migration_in: None, migration_out: None } kind = instances server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
54582025-04-30T06:43:59.554ZINFOtest_utilization (dropshot): request completed latency_us = 2939 local_addr = [::1]:39365 method = PUT remote_addr = [::1]:46906 req_id = bed4da72-ac57-454c-af79-b12d16034ded response_code = 200 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /vmms/9e8dbed7-faa3-4634-886b-3a005dd8e99b
54592025-04-30T06:43:59.556ZDEBGtest_utilization (nexus): client response SledAgent = b6d65341-167c-41df-9b5c-41cded99c229 result = Ok(Response { url: "http://[::1]:39365/vmms/9e8dbed7-faa3-4634-886b-3a005dd8e99b", status: 200, headers: {"content-type": "application/json", "x-request-id": "bed4da72-ac57-454c-af79-b12d16034ded", "content-length": "131", "date": "Wed, 30 Apr 2025 06:43:59 GMT"} })
54602025-04-30T06:43:59.556ZINFOtest_utilization (SagaExecutor): received new VMM runtime state from sled agent actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true migration_state = Migrations { migration_in: None, migration_out: None } propolis_id = 9e8dbed7-faa3-4634-886b-3a005dd8e99b saga_id = bc1ac890-e9e2-44d9-931e-3abe24cf75f5 saga_name = instance-start saga_node = EnsureRegistered vmm_state = VmmRuntimeState { state: Starting, gen: Generation(2), time_updated: 2025-04-30T06:43:59.549025164Z }
54612025-04-30T06:43:59.557ZTRACtest_utilization (SagaExecutor): authorize begin action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true resource = Database saga_id = bc1ac890-e9e2-44d9-931e-3abe24cf75f5 saga_name = instance-start saga_node = EnsureRegistered
54622025-04-30T06:43:59.557ZDEBGtest_utilization (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true roles = RoleSet { roles: {} } saga_id = bc1ac890-e9e2-44d9-931e-3abe24cf75f5 saga_name = instance-start saga_node = EnsureRegistered
54632025-04-30T06:43:59.559ZDEBGtest_utilization (SagaExecutor): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true resource = Database result = Ok(()) saga_id = bc1ac890-e9e2-44d9-931e-3abe24cf75f5 saga_name = instance-start saga_node = EnsureRegistered
54642025-04-30T06:43:59.575ZDEBGtest_utilization (SEC): saga log event new_state = N007 succeeded sec_id = ac55d8d9-a298-4534-898b-beee1c9e4e52
54652025-04-30T06:43:59.575ZDEBGtest_utilization (SecStore): recording saga event event_type = Succeeded(Object {"id": String("9e8dbed7-faa3-4634-886b-3a005dd8e99b"), "instance_id": String("d7bd2deb-ee27-4661-9e9a-524d734cd6ec"), "propolis_ip": String("::1:0/128"), "propolis_port": Number(12400), "runtime": Object {"gen": Number(2), "state": String("Starting"), "time_state_updated": String("2025-04-30T06:43:59.549025164Z")}, "sled_id": String("b6d65341-167c-41df-9b5c-41cded99c229"), "time_created": String("2025-04-30T06:43:58.339117Z"), "time_deleted": Null}) node_id = 7 saga_id = bc1ac890-e9e2-44d9-931e-3abe24cf75f5
54662025-04-30T06:43:59.585ZDEBGtest_utilization (SEC): saga log event new_state = N008 started sec_id = ac55d8d9-a298-4534-898b-beee1c9e4e52
54672025-04-30T06:43:59.585ZDEBGtest_utilization (SecStore): recording saga event event_type = Started node_id = 8 saga_id = bc1ac890-e9e2-44d9-931e-3abe24cf75f5
54682025-04-30T06:43:59.589ZTRACtest_utilization (SagaExecutor): authorize begin action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true resource = Database saga_id = bc1ac890-e9e2-44d9-931e-3abe24cf75f5 saga_name = instance-start saga_node = AddVirtualResources
54692025-04-30T06:43:59.589ZDEBGtest_utilization (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true roles = RoleSet { roles: {} } saga_id = bc1ac890-e9e2-44d9-931e-3abe24cf75f5 saga_name = instance-start saga_node = AddVirtualResources
54702025-04-30T06:43:59.590ZDEBGtest_utilization (SagaExecutor): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true resource = Database result = Ok(()) saga_id = bc1ac890-e9e2-44d9-931e-3abe24cf75f5 saga_name = instance-start saga_node = AddVirtualResources
54712025-04-30T06:43:59.608ZDEBGtest_utilization (SEC): saga log event new_state = N008 succeeded sec_id = ac55d8d9-a298-4534-898b-beee1c9e4e52
54722025-04-30T06:43:59.608ZDEBGtest_utilization (SecStore): recording saga event event_type = Succeeded(Null) node_id = 8 saga_id = bc1ac890-e9e2-44d9-931e-3abe24cf75f5
54732025-04-30T06:43:59.618ZDEBGtest_utilization (SEC): saga log event new_state = N009 started sec_id = ac55d8d9-a298-4534-898b-beee1c9e4e52
54742025-04-30T06:43:59.618ZDEBGtest_utilization (SecStore): recording saga event event_type = Started node_id = 9 saga_id = bc1ac890-e9e2-44d9-931e-3abe24cf75f5
54752025-04-30T06:43:59.624ZINFOtest_utilization (SagaExecutor): start saga: ensuring instance is running instance_id = d7bd2deb-ee27-4661-9e9a-524d734cd6ec saga_id = bc1ac890-e9e2-44d9-931e-3abe24cf75f5 saga_name = instance-start sled_id = b6d65341-167c-41df-9b5c-41cded99c229 start_reason = AutoStart
54762025-04-30T06:43:59.624ZTRACtest_utilization (InstanceAllocator): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000004ead, .. }) resource = Database
54772025-04-30T06:43:59.624ZDEBGtest_utilization (InstanceAllocator): roles roles = RoleSet { roles: {} }
54782025-04-30T06:43:59.625ZDEBGtest_utilization (InstanceAllocator): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000004ead, .. }) resource = Database result = Ok(())
54792025-04-30T06:43:59.629ZTRACtest_utilization (InstanceAllocator): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000004ead, .. }) resource = Sled { parent: Fleet, key: b6d65341-167c-41df-9b5c-41cded99c229, lookup_type: ById(b6d65341-167c-41df-9b5c-41cded99c229) }
54802025-04-30T06:43:59.629ZTRACtest_utilization (InstanceAllocator): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000004ead, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
54812025-04-30T06:43:59.629ZTRACtest_utilization (InstanceAllocator): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000004ead, .. }) resource = Database
54822025-04-30T06:43:59.629ZDEBGtest_utilization (InstanceAllocator): roles roles = RoleSet { roles: {} }
54832025-04-30T06:43:59.630ZDEBGtest_utilization (InstanceAllocator): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000004ead, .. }) resource = Database result = Ok(())
54842025-04-30T06:43:59.633ZDEBGtest_utilization (InstanceAllocator): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "viewer")} }
54852025-04-30T06:43:59.639ZDEBGtest_utilization (InstanceAllocator): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000004ead, .. }) resource = Sled { parent: Fleet, key: b6d65341-167c-41df-9b5c-41cded99c229, lookup_type: ById(b6d65341-167c-41df-9b5c-41cded99c229) } result = Ok(())
54862025-04-30T06:43:59.709ZDEBGtest_utilization (nexus): client request SledAgent = b6d65341-167c-41df-9b5c-41cded99c229 body = Some(Body) method = PUT uri = http://[::1]:39365/vmms/9e8dbed7-faa3-4634-886b-3a005dd8e99b/state
54872025-04-30T06:43:59.710ZINFOtest_utilization (dropshot): accepted connection local_addr = [::1]:39365 remote_addr = [::1]:52001 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
54882025-04-30T06:43:59.711ZTRACtest_utilization (dropshot): incoming request local_addr = [::1]:39365 method = PUT remote_addr = [::1]:52001 req_id = ca8a07d2-00dd-4cfd-8cce-fbcb90c8b5ba sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /vmms/9e8dbed7-faa3-4634-886b-3a005dd8e99b/state
54892025-04-30T06:43:59.712ZINFOtest_utilization (SledAgent): noop transition id = 9e8dbed7-faa3-4634-886b-3a005dd8e99b kind = instances server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 target = Running
54902025-04-30T06:43:59.712ZINFOtest_utilization (dropshot): request completed latency_us = 1017 local_addr = [::1]:39365 method = PUT remote_addr = [::1]:52001 req_id = ca8a07d2-00dd-4cfd-8cce-fbcb90c8b5ba response_code = 200 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /vmms/9e8dbed7-faa3-4634-886b-3a005dd8e99b/state
54912025-04-30T06:43:59.712ZDEBGtest_utilization (nexus): client response SledAgent = b6d65341-167c-41df-9b5c-41cded99c229 result = Ok(Response { url: "http://[::1]:39365/vmms/9e8dbed7-faa3-4634-886b-3a005dd8e99b/state", status: 200, headers: {"content-type": "application/json", "x-request-id": "ca8a07d2-00dd-4cfd-8cce-fbcb90c8b5ba", "content-length": "151", "date": "Wed, 30 Apr 2025 06:43:59 GMT"} })
54922025-04-30T06:43:59.713ZINFOtest_utilization (SagaExecutor): received new VMM runtime state from sled agent actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true migration_state = Migrations { migration_in: None, migration_out: None } propolis_id = 9e8dbed7-faa3-4634-886b-3a005dd8e99b saga_id = bc1ac890-e9e2-44d9-931e-3abe24cf75f5 saga_name = instance-start saga_node = EnsureRunning vmm_state = VmmRuntimeState { state: Starting, gen: Generation(2), time_updated: 2025-04-30T06:43:59.549025164Z }
54932025-04-30T06:43:59.713ZTRACtest_utilization (SagaExecutor): authorize begin action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true resource = Database saga_id = bc1ac890-e9e2-44d9-931e-3abe24cf75f5 saga_name = instance-start saga_node = EnsureRunning
54942025-04-30T06:43:59.713ZDEBGtest_utilization (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true roles = RoleSet { roles: {} } saga_id = bc1ac890-e9e2-44d9-931e-3abe24cf75f5 saga_name = instance-start saga_node = EnsureRunning
54952025-04-30T06:43:59.714ZDEBGtest_utilization (SagaExecutor): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true resource = Database result = Ok(()) saga_id = bc1ac890-e9e2-44d9-931e-3abe24cf75f5 saga_name = instance-start saga_node = EnsureRunning
54962025-04-30T06:43:59.764ZDEBGtest_utilization (SEC): saga log event new_state = N009 succeeded sec_id = ac55d8d9-a298-4534-898b-beee1c9e4e52
54972025-04-30T06:43:59.764ZDEBGtest_utilization (SecStore): recording saga event event_type = Succeeded(Null) node_id = 9 saga_id = bc1ac890-e9e2-44d9-931e-3abe24cf75f5
54982025-04-30T06:43:59.781ZDEBGtest_utilization (SEC): saga log event new_state = N011 started sec_id = ac55d8d9-a298-4534-898b-beee1c9e4e52
54992025-04-30T06:43:59.781ZDEBGtest_utilization (SecStore): recording saga event event_type = Started node_id = 11 saga_id = bc1ac890-e9e2-44d9-931e-3abe24cf75f5
55002025-04-30T06:43:59.799ZDEBGtest_utilization (SEC): saga log event new_state = N011 succeeded sec_id = ac55d8d9-a298-4534-898b-beee1c9e4e52
55012025-04-30T06:43:59.799ZDEBGtest_utilization (SecStore): recording saga event event_type = Succeeded(Null) node_id = 11 saga_id = bc1ac890-e9e2-44d9-931e-3abe24cf75f5
55022025-04-30T06:43:59.805ZINFOtest_utilization (SEC): update for saga cached state new_state = Done saga_id = bc1ac890-e9e2-44d9-931e-3abe24cf75f5 sec_id = ac55d8d9-a298-4534-898b-beee1c9e4e52
55032025-04-30T06:43:59.805ZINFOtest_utilization (SecStore): updating state new_state = done saga_id = bc1ac890-e9e2-44d9-931e-3abe24cf75f5
55042025-04-30T06:43:59.815ZINFOtest_utilization (SEC): saga finished result = success saga_id = bc1ac890-e9e2-44d9-931e-3abe24cf75f5 saga_name = instance-start sec_id = ac55d8d9-a298-4534-898b-beee1c9e4e52
55052025-04-30T06:43:59.816ZINFOtest_utilization (SagaExecutor): saga finished saga_id = bc1ac890-e9e2-44d9-931e-3abe24cf75f5 saga_name = instance-start saga_result = SagaResult { saga_id: bc1ac890-e9e2-44d9-931e-3abe24cf75f5, saga_log: SagaLog { saga_id: bc1ac890-e9e2-44d9-931e-3abe24cf75f5, unwinding: false, events: [N010 started, N010 succeeded, N000 started, N000 succeeded, N001 started, N001 succeeded, N002 started, N002 succeeded, N003 started, N003 succeeded, N004 started, N004 succeeded, N005 started, N005 succeeded, N006 started, N006 succeeded, N007 started, N007 succeeded, N008 started, N008 succeeded, N009 started, N009 succeeded, N011 started, N011 succeeded], node_status: {0: Succeeded(String("9e8dbed7-faa3-4634-886b-3a005dd8e99b")), 1: Succeeded(String("b6d65341-167c-41df-9b5c-41cded99c229")), 2: Succeeded(String("::1:0")), 3: Succeeded(Object {"id": String("9e8dbed7-faa3-4634-886b-3a005dd8e99b"), "instance_id": String("d7bd2deb-ee27-4661-9e9a-524d734cd6ec"), "propolis_ip": String("::1:0/128"), "propolis_port": Number(12400), "runtime": Object {"gen": Number(1), "state": String("Creating"), "time_state_updated": String("2025-04-30T06:43:58.339117Z")}, "sled_id": String("b6d65341-167c-41df-9b5c-41cded99c229"), "time_created": String("2025-04-30T06:43:58.339117Z"), "time_deleted": Null}), 4: Succeeded(Object {"auto_restart": Object {"cooldown": Null, "policy": Null}, "boot_disk_id": Null, "hostname": String("the-host"), "identity": Object {"description": String("instance \\"utilization-test-instance\\""), "id": String("d7bd2deb-ee27-4661-9e9a-524d734cd6ec"), "name": String("utilization-test-instance"), "time_created": String("2025-04-30T06:43:54.249576Z"), "time_deleted": Null, "time_modified": String("2025-04-30T06:43:54.249576Z")}, "memory": Number(1073741824), "ncpus": Number(4), "project_id": String("473b39f9-8669-45b3-b3d7-793f7e10fc31"), "runtime_state": Object {"dst_propolis_id": Null, "gen": Number(3), "migration_id": Null, "nexus_state": String("Vmm"), "propolis_id": String("9e8dbed7-faa3-4634-886b-3a005dd8e99b"), "time_last_auto_restarted": Null, "time_updated": String("2025-04-30T06:43:54.249576Z")}, "updater_gen": Number(1), "updater_id": Null, "user_data": Array [Number(35), Number(99), Number(108), Number(111), Number(117), Number(100), Number(45), Number(99), Number(111), Number(110), Number(102), Number(105), Number(103), Number(10), Number(115), Number(121), Number(115), Number(116), Number(101), Number(109), Number(95), Number(105), Number(110), Number(102), Number(111), Number(58), Number(10), Number(32), Number(32), Number(100), Number(101), Number(102), Number(97), Number(117), Number(108), Number(116), Number(95), Number(117), Number(115), Number(101), Number(114), Number(58), Number(10), Number(32), Number(32), Number(32), Number(32), Number(110), Number(97), Number(109), Number(101), Number(58), Number(32), Number(111), Number(120), Number(105), Number(100), Number(101)]}), 5: Succeeded(Null), 6: Succeeded(Null), 7: Succeeded(Object {"id": String("9e8dbed7-faa3-4634-886b-3a005dd8e99b"), "instance_id": String("d7bd2deb-ee27-4661-9e9a-524d734cd6ec"), "propolis_ip": String("::1:0/128"), "propolis_port": Number(12400), "runtime": Object {"gen": Number(2), "state": String("Starting"), "time_state_updated": String("2025-04-30T06:43:59.549025164Z")}, "sled_id": String("b6d65341-167c-41df-9b5c-41cded99c229"), "time_created": String("2025-04-30T06:43:58.339117Z"), "time_deleted": Null}), 8: Succeeded(Null), 9: Succeeded(Null), 10: Succeeded(Null), 11: Succeeded(Null)} }, kind: Ok(SagaResultOk { saga_output: Null, node_outputs: {"dpd_ensure": Null, "ensure_registered": Object {"id": String("9e8dbed7-faa3-4634-886b-3a005dd8e99b"), "instance_id": String("d7bd2deb-ee27-4661-9e9a-524d734cd6ec"), "propolis_ip": String("::1:0/128"), "propolis_port": Number(12400), "runtime": Object {"gen": Number(2), "state": String("Starting"), "time_state_updated": String("2025-04-30T06:43:59.549025164Z")}, "sled_id": String("b6d65341-167c-41df-9b5c-41cded99c229"), "time_created": String("2025-04-30T06:43:58.339117Z"), "time_deleted": Null}, "ensure_running": Null, "propolis_id": String("9e8dbed7-faa3-4634-886b-3a005dd8e99b"), "propolis_ip": String("::1:0"), "sled_id": String("b6d65341-167c-41df-9b5c-41cded99c229"), "started_record": Object {"auto_restart": Object {"cooldown": Null, "policy": Null}, "boot_disk_id": Null, "hostname": String("the-host"), "identity": Object {"description": String("instance \\"utilization-test-instance\\""), "id": String("d7bd2deb-ee27-4661-9e9a-524d734cd6ec"), "name": String("utilization-test-instance"), "time_created": String("2025-04-30T06:43:54.249576Z"), "time_deleted": Null, "time_modified": String("2025-04-30T06:43:54.249576Z")}, "memory": Number(1073741824), "ncpus": Number(4), "project_id": String("473b39f9-8669-45b3-b3d7-793f7e10fc31"), "runtime_state": Object {"dst_propolis_id": Null, "gen": Number(3), "migration_id": Null, "nexus_state": String("Vmm"), "propolis_id": String("9e8dbed7-faa3-4634-886b-3a005dd8e99b"), "time_last_auto_restarted": Null, "time_updated": String("2025-04-30T06:43:54.249576Z")}, "updater_gen": Number(1), "updater_id": Null, "user_data": Array [Number(35), Number(99), Number(108), Number(111), Number(117), Number(100), Number(45), Number(99), Number(111), Number(110), Number(102), Number(105), Number(103), Number(10), Number(115), Number(121), Number(115), Number(116), Number(101), Number(109), Number(95), Number(105), Number(110), Number(102), Number(111), Number(58), Number(10), Number(32), Number(32), Number(100), Number(101), Number(102), Number(97), Number(117), Number(108), Number(116), Number(95), Number(117), Number(115), Number(101), Number(114), Number(58), Number(10), Number(32), Number(32), Number(32), Number(32), Number(110), Number(97), Number(109), Number(101), Number(58), Number(32), Number(111), Number(120), Number(105), Number(100), Number(101)]}, "v2p_ensure": Null, "virtual_resources": Null, "vmm_record": Object {"id": String("9e8dbed7-faa3-4634-886b-3a005dd8e99b"), "instance_id": String("d7bd2deb-ee27-4661-9e9a-524d734cd6ec"), "propolis_ip": String("::1:0/128"), "propolis_port": Number(12400), "runtime": Object {"gen": Number(1), "state": String("Creating"), "time_state_updated": String("2025-04-30T06:43:58.339117Z")}, "sled_id": String("b6d65341-167c-41df-9b5c-41cded99c229"), "time_created": String("2025-04-30T06:43:58.339117Z"), "time_deleted": Null}} }) }
55062025-04-30T06:43:59.816ZTRACtest_utilization (dropshot_external): authorize begin action = Read actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true local_addr = 127.0.0.1:46348 method = POST remote_addr = 127.0.0.1:41283 req_id = bf272e5e-cbe5-41d9-b505-34877ffe12d2 resource = Instance { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000000, lookup_type: ById(001de000-5110-4000-8000-000000000000) }, key: 473b39f9-8669-45b3-b3d7-793f7e10fc31, lookup_type: ById(473b39f9-8669-45b3-b3d7-793f7e10fc31) }, key: d7bd2deb-ee27-4661-9e9a-524d734cd6ec, lookup_type: ById(d7bd2deb-ee27-4661-9e9a-524d734cd6ec) } uri = /v1/instances?project=utilization-test-project
55072025-04-30T06:43:59.816ZTRACtest_utilization (dropshot_external): loading roles actor = Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. } actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true local_addr = 127.0.0.1:46348 method = POST remote_addr = 127.0.0.1:41283 req_id = bf272e5e-cbe5-41d9-b505-34877ffe12d2 resource_id = 473b39f9-8669-45b3-b3d7-793f7e10fc31 resource_type = Project uri = /v1/instances?project=utilization-test-project
55082025-04-30T06:43:59.816ZTRACtest_utilization (dropshot_external): authorize begin action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true local_addr = 127.0.0.1:46348 method = POST remote_addr = 127.0.0.1:41283 req_id = bf272e5e-cbe5-41d9-b505-34877ffe12d2 resource = Database uri = /v1/instances?project=utilization-test-project
55092025-04-30T06:43:59.816ZDEBGtest_utilization (dropshot_external): roles actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true local_addr = 127.0.0.1:46348 method = POST remote_addr = 127.0.0.1:41283 req_id = bf272e5e-cbe5-41d9-b505-34877ffe12d2 roles = RoleSet { roles: {} } uri = /v1/instances?project=utilization-test-project
55102025-04-30T06:43:59.817ZDEBGtest_utilization (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true local_addr = 127.0.0.1:46348 method = POST remote_addr = 127.0.0.1:41283 req_id = bf272e5e-cbe5-41d9-b505-34877ffe12d2 resource = Database result = Ok(()) uri = /v1/instances?project=utilization-test-project
55112025-04-30T06:43:59.826ZTRACtest_utilization (dropshot_external): loading roles actor = Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. } actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true local_addr = 127.0.0.1:46348 method = POST remote_addr = 127.0.0.1:41283 req_id = bf272e5e-cbe5-41d9-b505-34877ffe12d2 resource_id = 001de000-5110-4000-8000-000000000000 resource_type = Silo uri = /v1/instances?project=utilization-test-project
55122025-04-30T06:43:59.826ZTRACtest_utilization (dropshot_external): authorize begin action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true local_addr = 127.0.0.1:46348 method = POST remote_addr = 127.0.0.1:41283 req_id = bf272e5e-cbe5-41d9-b505-34877ffe12d2 resource = Database uri = /v1/instances?project=utilization-test-project
55132025-04-30T06:43:59.826ZDEBGtest_utilization (dropshot_external): roles actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true local_addr = 127.0.0.1:46348 method = POST remote_addr = 127.0.0.1:41283 req_id = bf272e5e-cbe5-41d9-b505-34877ffe12d2 roles = RoleSet { roles: {} } uri = /v1/instances?project=utilization-test-project
55142025-04-30T06:43:59.827ZDEBGtest_utilization (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true local_addr = 127.0.0.1:46348 method = POST remote_addr = 127.0.0.1:41283 req_id = bf272e5e-cbe5-41d9-b505-34877ffe12d2 resource = Database result = Ok(()) uri = /v1/instances?project=utilization-test-project
55152025-04-30T06:43:59.830ZTRACtest_utilization (dropshot_external): loading roles actor = Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. } actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true local_addr = 127.0.0.1:46348 method = POST remote_addr = 127.0.0.1:41283 req_id = bf272e5e-cbe5-41d9-b505-34877ffe12d2 resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet uri = /v1/instances?project=utilization-test-project
55162025-04-30T06:43:59.830ZTRACtest_utilization (dropshot_external): authorize begin action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true local_addr = 127.0.0.1:46348 method = POST remote_addr = 127.0.0.1:41283 req_id = bf272e5e-cbe5-41d9-b505-34877ffe12d2 resource = Database uri = /v1/instances?project=utilization-test-project
55172025-04-30T06:43:59.830ZDEBGtest_utilization (dropshot_external): roles actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true local_addr = 127.0.0.1:46348 method = POST remote_addr = 127.0.0.1:41283 req_id = bf272e5e-cbe5-41d9-b505-34877ffe12d2 roles = RoleSet { roles: {} } uri = /v1/instances?project=utilization-test-project
55182025-04-30T06:43:59.831ZDEBGtest_utilization (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true local_addr = 127.0.0.1:46348 method = POST remote_addr = 127.0.0.1:41283 req_id = bf272e5e-cbe5-41d9-b505-34877ffe12d2 resource = Database result = Ok(()) uri = /v1/instances?project=utilization-test-project
55192025-04-30T06:43:59.844ZDEBGtest_utilization (dropshot_external): roles actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true local_addr = 127.0.0.1:46348 method = POST remote_addr = 127.0.0.1:41283 req_id = bf272e5e-cbe5-41d9-b505-34877ffe12d2 roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin"), (Silo, 001de000-5110-4000-8000-000000000000, "admin")} } uri = /v1/instances?project=utilization-test-project
55202025-04-30T06:43:59.844ZDEBGtest_utilization (dropshot_external): authorize result action = Read actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true local_addr = 127.0.0.1:46348 method = POST remote_addr = 127.0.0.1:41283 req_id = bf272e5e-cbe5-41d9-b505-34877ffe12d2 resource = Instance { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000000, lookup_type: ById(001de000-5110-4000-8000-000000000000) }, key: 473b39f9-8669-45b3-b3d7-793f7e10fc31, lookup_type: ById(473b39f9-8669-45b3-b3d7-793f7e10fc31) }, key: d7bd2deb-ee27-4661-9e9a-524d734cd6ec, lookup_type: ById(d7bd2deb-ee27-4661-9e9a-524d734cd6ec) } result = Ok(()) uri = /v1/instances?project=utilization-test-project
55212025-04-30T06:43:59.844ZTRACtest_utilization (dropshot_external): authorize begin action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true local_addr = 127.0.0.1:46348 method = POST remote_addr = 127.0.0.1:41283 req_id = bf272e5e-cbe5-41d9-b505-34877ffe12d2 resource = Database uri = /v1/instances?project=utilization-test-project
55222025-04-30T06:43:59.844ZDEBGtest_utilization (dropshot_external): roles actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true local_addr = 127.0.0.1:46348 method = POST remote_addr = 127.0.0.1:41283 req_id = bf272e5e-cbe5-41d9-b505-34877ffe12d2 roles = RoleSet { roles: {} } uri = /v1/instances?project=utilization-test-project
55232025-04-30T06:43:59.844ZDEBGtest_utilization (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true local_addr = 127.0.0.1:46348 method = POST remote_addr = 127.0.0.1:41283 req_id = bf272e5e-cbe5-41d9-b505-34877ffe12d2 resource = Database result = Ok(()) uri = /v1/instances?project=utilization-test-project
55242025-04-30T06:43:59.852ZTRACtest_utilization (dropshot_external): authorize begin action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true local_addr = 127.0.0.1:46348 method = POST remote_addr = 127.0.0.1:41283 req_id = bf272e5e-cbe5-41d9-b505-34877ffe12d2 resource = Database uri = /v1/instances?project=utilization-test-project
55252025-04-30T06:43:59.852ZDEBGtest_utilization (dropshot_external): roles actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true local_addr = 127.0.0.1:46348 method = POST remote_addr = 127.0.0.1:41283 req_id = bf272e5e-cbe5-41d9-b505-34877ffe12d2 roles = RoleSet { roles: {} } uri = /v1/instances?project=utilization-test-project
55262025-04-30T06:43:59.853ZDEBGtest_utilization (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true local_addr = 127.0.0.1:46348 method = POST remote_addr = 127.0.0.1:41283 req_id = bf272e5e-cbe5-41d9-b505-34877ffe12d2 resource = Database result = Ok(()) uri = /v1/instances?project=utilization-test-project
55272025-04-30T06:43:59.853ZDEBGtest_utilization (BackgroundTasks): activating background_task = vpc_route_manager iteration = 2 reason = Signaled
55282025-04-30T06:43:59.853ZINFOtest_utilization (BackgroundTasks): VPC route manager running background_task = vpc_route_manager
55292025-04-30T06:43:59.853ZTRACtest_utilization (BackgroundTasks): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet
55302025-04-30T06:43:59.853ZTRACtest_utilization (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = vpc_route_manager resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
55312025-04-30T06:43:59.853ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database
55322025-04-30T06:43:59.853ZDEBGtest_utilization (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
55332025-04-30T06:43:59.854ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
55342025-04-30T06:43:59.858ZTRACtest_utilization (dropshot_external): authorize begin action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true local_addr = 127.0.0.1:46348 method = POST remote_addr = 127.0.0.1:41283 req_id = bf272e5e-cbe5-41d9-b505-34877ffe12d2 resource = Database uri = /v1/instances?project=utilization-test-project
55352025-04-30T06:43:59.859ZDEBGtest_utilization (dropshot_external): roles actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true local_addr = 127.0.0.1:46348 method = POST remote_addr = 127.0.0.1:41283 req_id = bf272e5e-cbe5-41d9-b505-34877ffe12d2 roles = RoleSet { roles: {} } uri = /v1/instances?project=utilization-test-project
55362025-04-30T06:43:59.860ZDEBGtest_utilization (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true local_addr = 127.0.0.1:46348 method = POST remote_addr = 127.0.0.1:41283 req_id = bf272e5e-cbe5-41d9-b505-34877ffe12d2 resource = Database result = Ok(()) uri = /v1/instances?project=utilization-test-project
55372025-04-30T06:43:59.860ZDEBGtest_utilization (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
55382025-04-30T06:43:59.872ZDEBGtest_utilization (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
55392025-04-30T06:43:59.872ZTRACtest_utilization (BackgroundTasks): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet
55402025-04-30T06:43:59.872ZTRACtest_utilization (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = vpc_route_manager resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
55412025-04-30T06:43:59.872ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database
55422025-04-30T06:43:59.872ZDEBGtest_utilization (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
55432025-04-30T06:43:59.873ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
55442025-04-30T06:43:59.875ZTRACtest_utilization (dropshot_external): authorize begin action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true local_addr = 127.0.0.1:46348 method = POST remote_addr = 127.0.0.1:41283 req_id = bf272e5e-cbe5-41d9-b505-34877ffe12d2 resource = Database uri = /v1/instances?project=utilization-test-project
55452025-04-30T06:43:59.875ZDEBGtest_utilization (dropshot_external): roles actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true local_addr = 127.0.0.1:46348 method = POST remote_addr = 127.0.0.1:41283 req_id = bf272e5e-cbe5-41d9-b505-34877ffe12d2 roles = RoleSet { roles: {} } uri = /v1/instances?project=utilization-test-project
55462025-04-30T06:43:59.876ZDEBGtest_utilization (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true local_addr = 127.0.0.1:46348 method = POST remote_addr = 127.0.0.1:41283 req_id = bf272e5e-cbe5-41d9-b505-34877ffe12d2 resource = Database result = Ok(()) uri = /v1/instances?project=utilization-test-project
55472025-04-30T06:43:59.877ZDEBGtest_utilization (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
55482025-04-30T06:43:59.882ZDEBGtest_utilization (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
55492025-04-30T06:43:59.882ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database
55502025-04-30T06:43:59.882ZDEBGtest_utilization (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
55512025-04-30T06:43:59.883ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
55522025-04-30T06:43:59.885ZTRACtest_utilization (dropshot_external): authorize begin action = Read actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true local_addr = 127.0.0.1:46348 method = POST remote_addr = 127.0.0.1:41283 req_id = bf272e5e-cbe5-41d9-b505-34877ffe12d2 resource = Instance { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000000, lookup_type: ById(001de000-5110-4000-8000-000000000000) }, key: 473b39f9-8669-45b3-b3d7-793f7e10fc31, lookup_type: ById(473b39f9-8669-45b3-b3d7-793f7e10fc31) }, key: d7bd2deb-ee27-4661-9e9a-524d734cd6ec, lookup_type: ById(d7bd2deb-ee27-4661-9e9a-524d734cd6ec) } uri = /v1/instances?project=utilization-test-project
55532025-04-30T06:43:59.885ZTRACtest_utilization (dropshot_external): loading roles actor = Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. } actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true local_addr = 127.0.0.1:46348 method = POST remote_addr = 127.0.0.1:41283 req_id = bf272e5e-cbe5-41d9-b505-34877ffe12d2 resource_id = 473b39f9-8669-45b3-b3d7-793f7e10fc31 resource_type = Project uri = /v1/instances?project=utilization-test-project
55542025-04-30T06:43:59.886ZTRACtest_utilization (dropshot_external): authorize begin action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true local_addr = 127.0.0.1:46348 method = POST remote_addr = 127.0.0.1:41283 req_id = bf272e5e-cbe5-41d9-b505-34877ffe12d2 resource = Database uri = /v1/instances?project=utilization-test-project
55552025-04-30T06:43:59.886ZDEBGtest_utilization (dropshot_external): roles actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true local_addr = 127.0.0.1:46348 method = POST remote_addr = 127.0.0.1:41283 req_id = bf272e5e-cbe5-41d9-b505-34877ffe12d2 roles = RoleSet { roles: {} } uri = /v1/instances?project=utilization-test-project
55562025-04-30T06:43:59.887ZDEBGtest_utilization (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true local_addr = 127.0.0.1:46348 method = POST remote_addr = 127.0.0.1:41283 req_id = bf272e5e-cbe5-41d9-b505-34877ffe12d2 resource = Database result = Ok(()) uri = /v1/instances?project=utilization-test-project
55572025-04-30T06:43:59.992ZINFOtest_utilization (BackgroundTasks): VPC route manager sled b6d65341-167c-41df-9b5c-41cded99c229 background_task = vpc_route_manager
55582025-04-30T06:43:59.993ZDEBGtest_utilization (BackgroundTasks): client request SledAgent = b6d65341-167c-41df-9b5c-41cded99c229 background_task = vpc_route_manager body = None method = GET uri = http://[::1]:39365/vpc-routes
55592025-04-30T06:43:59.996ZTRACtest_utilization (dropshot_external): loading roles actor = Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. } actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true local_addr = 127.0.0.1:46348 method = POST remote_addr = 127.0.0.1:41283 req_id = bf272e5e-cbe5-41d9-b505-34877ffe12d2 resource_id = 001de000-5110-4000-8000-000000000000 resource_type = Silo uri = /v1/instances?project=utilization-test-project
55602025-04-30T06:43:59.996ZTRACtest_utilization (dropshot_external): authorize begin action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true local_addr = 127.0.0.1:46348 method = POST remote_addr = 127.0.0.1:41283 req_id = bf272e5e-cbe5-41d9-b505-34877ffe12d2 resource = Database uri = /v1/instances?project=utilization-test-project
55612025-04-30T06:43:59.996ZDEBGtest_utilization (dropshot_external): roles actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true local_addr = 127.0.0.1:46348 method = POST remote_addr = 127.0.0.1:41283 req_id = bf272e5e-cbe5-41d9-b505-34877ffe12d2 roles = RoleSet { roles: {} } uri = /v1/instances?project=utilization-test-project
55622025-04-30T06:43:59.997ZDEBGtest_utilization (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true local_addr = 127.0.0.1:46348 method = POST remote_addr = 127.0.0.1:41283 req_id = bf272e5e-cbe5-41d9-b505-34877ffe12d2 resource = Database result = Ok(()) uri = /v1/instances?project=utilization-test-project
55632025-04-30T06:43:59.998ZINFOtest_utilization (dropshot): accepted connection local_addr = [::1]:39365 remote_addr = [::1]:33244 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
55642025-04-30T06:43:59.999ZINFOtest_utilization (oximeter-agent): refreshing list of producers from Nexus collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 file = oximeter/collector/src/agent.rs:562
55652025-04-30T06:43:59.999ZDEBGtest_utilization (collection-task): sent timer-based collection request to the collection task collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
55662025-04-30T06:43:59.999ZDEBGtest_utilization (collection-task): collecting from producer collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
55672025-04-30T06:44:00.000ZDEBGtest_utilization (oximeter): client request body = None method = GET uri = http://127.0.0.1:32956/metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?sort_by=id_ascending
55682025-04-30T06:44:00.001ZTRACtest_utilization (dropshot_internal): incoming request local_addr = 127.0.0.1:32956 method = GET remote_addr = 127.0.0.1:55797 req_id = 7a7f45ef-85be-448e-9959-3382e18ada73 uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?sort_by=id_ascending
55692025-04-30T06:44:00.002ZTRACtest_utilization (dropshot): incoming request local_addr = [::1]:39365 method = GET remote_addr = [::1]:33244 req_id = b61b603d-fa36-477b-a8a4-ebdcd821963c sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /vpc-routes
55702025-04-30T06:44:00.002ZTRACtest_utilization (dropshot_internal): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = 127.0.0.1:32956 method = GET remote_addr = 127.0.0.1:55797 req_id = 7a7f45ef-85be-448e-9959-3382e18ada73 resource = Database uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?sort_by=id_ascending
55712025-04-30T06:44:00.002ZDEBGtest_utilization (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = 127.0.0.1:32956 method = GET remote_addr = 127.0.0.1:55797 req_id = 7a7f45ef-85be-448e-9959-3382e18ada73 roles = RoleSet { roles: {} } uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?sort_by=id_ascending
55722025-04-30T06:44:00.003ZDEBGtest_utilization (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = 127.0.0.1:32956 method = GET remote_addr = 127.0.0.1:55797 req_id = 7a7f45ef-85be-448e-9959-3382e18ada73 resource = Database result = Ok(()) uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?sort_by=id_ascending
55732025-04-30T06:44:00.005ZINFOtest_utilization (dropshot): request completed latency_us = 3840 local_addr = [::1]:39365 method = GET remote_addr = [::1]:33244 req_id = b61b603d-fa36-477b-a8a4-ebdcd821963c response_code = 200 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /vpc-routes
55742025-04-30T06:44:00.008ZTRACtest_utilization (dropshot_external): loading roles actor = Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. } actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true local_addr = 127.0.0.1:46348 method = POST remote_addr = 127.0.0.1:41283 req_id = bf272e5e-cbe5-41d9-b505-34877ffe12d2 resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet uri = /v1/instances?project=utilization-test-project
55752025-04-30T06:44:00.008ZTRACtest_utilization (dropshot_external): authorize begin action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true local_addr = 127.0.0.1:46348 method = POST remote_addr = 127.0.0.1:41283 req_id = bf272e5e-cbe5-41d9-b505-34877ffe12d2 resource = Database uri = /v1/instances?project=utilization-test-project
55762025-04-30T06:44:00.008ZDEBGtest_utilization (dropshot_external): roles actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true local_addr = 127.0.0.1:46348 method = POST remote_addr = 127.0.0.1:41283 req_id = bf272e5e-cbe5-41d9-b505-34877ffe12d2 roles = RoleSet { roles: {} } uri = /v1/instances?project=utilization-test-project
55772025-04-30T06:44:00.014ZDEBGtest_utilization (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true local_addr = 127.0.0.1:46348 method = POST remote_addr = 127.0.0.1:41283 req_id = bf272e5e-cbe5-41d9-b505-34877ffe12d2 resource = Database result = Ok(()) uri = /v1/instances?project=utilization-test-project
55782025-04-30T06:44:00.016ZTRACtest_utilization (collection-task): sent collection request to producer collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
55792025-04-30T06:44:00.017ZDEBGtest_utilization (collection-task): collected results from producer collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 n_results = 1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
55802025-04-30T06:44:00.017ZTRACtest_utilization (collection-task): forwarded results to main collection loop collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
55812025-04-30T06:44:00.017ZTRACtest_utilization (collection-task): top of inner collection loop, waiting for next request collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
55822025-04-30T06:44:00.022ZDEBGtest_utilization (BackgroundTasks): client response SledAgent = b6d65341-167c-41df-9b5c-41cded99c229 background_task = vpc_route_manager result = Ok(Response { url: "http://[::1]:39365/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "b61b603d-fa36-477b-a8a4-ebdcd821963c", "content-length": "154", "date": "Wed, 30 Apr 2025 06:43:59 GMT"} })
55832025-04-30T06:44:00.023ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database
55842025-04-30T06:44:00.023ZDEBGtest_utilization (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
55852025-04-30T06:44:00.023ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
55862025-04-30T06:44:00.024ZINFOtest_utilization (dropshot_internal): request completed latency_us = 23128 local_addr = 127.0.0.1:32956 method = GET remote_addr = 127.0.0.1:55797 req_id = 7a7f45ef-85be-448e-9959-3382e18ada73 response_code = 200 uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?sort_by=id_ascending
55872025-04-30T06:44:00.026ZDEBGtest_utilization (oximeter): client response result = Ok(Response { url: "http://127.0.0.1:32956/metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?sort_by=id_ascending", status: 200, headers: {"content-type": "application/json", "x-request-id": "7a7f45ef-85be-448e-9959-3382e18ada73", "content-length": "405", "date": "Wed, 30 Apr 2025 06:43:59 GMT"} })
55882025-04-30T06:44:00.037ZDEBGtest_utilization (oximeter): client request body = None method = GET uri = http://127.0.0.1:32956/metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJhYzU1ZDhkOS1hMjk4LTQ1MzQtODk4Yi1iZWVlMWM5ZTRlNTIifX0%3D
55892025-04-30T06:44:00.037ZDEBGtest_utilization (dropshot_external): roles actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true local_addr = 127.0.0.1:46348 method = POST remote_addr = 127.0.0.1:41283 req_id = bf272e5e-cbe5-41d9-b505-34877ffe12d2 roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin"), (Silo, 001de000-5110-4000-8000-000000000000, "admin")} } uri = /v1/instances?project=utilization-test-project
55902025-04-30T06:44:00.050ZDEBGtest_utilization (dropshot_external): authorize result action = Read actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true local_addr = 127.0.0.1:46348 method = POST remote_addr = 127.0.0.1:41283 req_id = bf272e5e-cbe5-41d9-b505-34877ffe12d2 resource = Instance { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000000, lookup_type: ById(001de000-5110-4000-8000-000000000000) }, key: 473b39f9-8669-45b3-b3d7-793f7e10fc31, lookup_type: ById(473b39f9-8669-45b3-b3d7-793f7e10fc31) }, key: d7bd2deb-ee27-4661-9e9a-524d734cd6ec, lookup_type: ById(d7bd2deb-ee27-4661-9e9a-524d734cd6ec) } result = Ok(()) uri = /v1/instances?project=utilization-test-project
55912025-04-30T06:44:00.050ZTRACtest_utilization (dropshot_external): authorize begin action = Read actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true local_addr = 127.0.0.1:46348 method = POST remote_addr = 127.0.0.1:41283 req_id = bf272e5e-cbe5-41d9-b505-34877ffe12d2 resource = Instance { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000000, lookup_type: ById(001de000-5110-4000-8000-000000000000) }, key: 473b39f9-8669-45b3-b3d7-793f7e10fc31, lookup_type: ById(473b39f9-8669-45b3-b3d7-793f7e10fc31) }, key: d7bd2deb-ee27-4661-9e9a-524d734cd6ec, lookup_type: ById(d7bd2deb-ee27-4661-9e9a-524d734cd6ec) } uri = /v1/instances?project=utilization-test-project
55922025-04-30T06:44:00.051ZTRACtest_utilization (dropshot_external): loading roles actor = Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. } actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true local_addr = 127.0.0.1:46348 method = POST remote_addr = 127.0.0.1:41283 req_id = bf272e5e-cbe5-41d9-b505-34877ffe12d2 resource_id = 473b39f9-8669-45b3-b3d7-793f7e10fc31 resource_type = Project uri = /v1/instances?project=utilization-test-project
55932025-04-30T06:44:00.051ZTRACtest_utilization (dropshot_external): authorize begin action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true local_addr = 127.0.0.1:46348 method = POST remote_addr = 127.0.0.1:41283 req_id = bf272e5e-cbe5-41d9-b505-34877ffe12d2 resource = Database uri = /v1/instances?project=utilization-test-project
55942025-04-30T06:44:00.051ZDEBGtest_utilization (dropshot_external): roles actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true local_addr = 127.0.0.1:46348 method = POST remote_addr = 127.0.0.1:41283 req_id = bf272e5e-cbe5-41d9-b505-34877ffe12d2 roles = RoleSet { roles: {} } uri = /v1/instances?project=utilization-test-project
55952025-04-30T06:44:00.051ZDEBGtest_utilization (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true local_addr = 127.0.0.1:46348 method = POST remote_addr = 127.0.0.1:41283 req_id = bf272e5e-cbe5-41d9-b505-34877ffe12d2 resource = Database result = Ok(()) uri = /v1/instances?project=utilization-test-project
55962025-04-30T06:44:00.052ZTRACtest_utilization (dropshot_internal): incoming request local_addr = 127.0.0.1:32956 method = GET remote_addr = 127.0.0.1:55797 req_id = 48c95d0c-6981-4e26-9152-d90e27ea3ef0 uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJhYzU1ZDhkOS1hMjk4LTQ1MzQtODk4Yi1iZWVlMWM5ZTRlNTIifX0%3D
55972025-04-30T06:44:00.052ZTRACtest_utilization (dropshot_internal): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = 127.0.0.1:32956 method = GET remote_addr = 127.0.0.1:55797 req_id = 48c95d0c-6981-4e26-9152-d90e27ea3ef0 resource = Database uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJhYzU1ZDhkOS1hMjk4LTQ1MzQtODk4Yi1iZWVlMWM5ZTRlNTIifX0%3D
55982025-04-30T06:44:00.053ZDEBGtest_utilization (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = 127.0.0.1:32956 method = GET remote_addr = 127.0.0.1:55797 req_id = 48c95d0c-6981-4e26-9152-d90e27ea3ef0 roles = RoleSet { roles: {} } uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJhYzU1ZDhkOS1hMjk4LTQ1MzQtODk4Yi1iZWVlMWM5ZTRlNTIifX0%3D
55992025-04-30T06:44:00.055ZDEBGtest_utilization (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = 127.0.0.1:32956 method = GET remote_addr = 127.0.0.1:55797 req_id = 48c95d0c-6981-4e26-9152-d90e27ea3ef0 resource = Database result = Ok(()) uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJhYzU1ZDhkOS1hMjk4LTQ1MzQtODk4Yi1iZWVlMWM5ZTRlNTIifX0%3D
56002025-04-30T06:44:00.056ZINFOtest_utilization (BackgroundTasks): computed internet gateway mappings for sled assocs = {64c48731-6cc1-46ff-99f2-86f6c481d0c6: {10.0.0.0: {5567c57c-3b46-4ca0-968f-03632bf48df0}}} background_task = vpc_route_manager sled = sim-b6d65341
56012025-04-30T06:44:00.056ZDEBGtest_utilization (BackgroundTasks): client request SledAgent = b6d65341-167c-41df-9b5c-41cded99c229 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[::1]:39365/eip-gateways
56022025-04-30T06:44:00.056ZTRACtest_utilization (dropshot): incoming request local_addr = [::1]:39365 method = PUT remote_addr = [::1]:33244 req_id = cffb43fc-c9c4-468b-b86d-ca9627669018 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /eip-gateways
56032025-04-30T06:44:00.057ZINFOtest_utilization (dropshot): request completed latency_us = 408 local_addr = [::1]:39365 method = PUT remote_addr = [::1]:33244 req_id = cffb43fc-c9c4-468b-b86d-ca9627669018 response_code = 204 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /eip-gateways
56042025-04-30T06:44:00.058ZTRACtest_utilization (dropshot_external): loading roles actor = Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. } actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true local_addr = 127.0.0.1:46348 method = POST remote_addr = 127.0.0.1:41283 req_id = bf272e5e-cbe5-41d9-b505-34877ffe12d2 resource_id = 001de000-5110-4000-8000-000000000000 resource_type = Silo uri = /v1/instances?project=utilization-test-project
56052025-04-30T06:44:00.058ZTRACtest_utilization (dropshot_external): authorize begin action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true local_addr = 127.0.0.1:46348 method = POST remote_addr = 127.0.0.1:41283 req_id = bf272e5e-cbe5-41d9-b505-34877ffe12d2 resource = Database uri = /v1/instances?project=utilization-test-project
56062025-04-30T06:44:00.058ZDEBGtest_utilization (dropshot_external): roles actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true local_addr = 127.0.0.1:46348 method = POST remote_addr = 127.0.0.1:41283 req_id = bf272e5e-cbe5-41d9-b505-34877ffe12d2 roles = RoleSet { roles: {} } uri = /v1/instances?project=utilization-test-project
56072025-04-30T06:44:00.058ZDEBGtest_utilization (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true local_addr = 127.0.0.1:46348 method = POST remote_addr = 127.0.0.1:41283 req_id = bf272e5e-cbe5-41d9-b505-34877ffe12d2 resource = Database result = Ok(()) uri = /v1/instances?project=utilization-test-project
56082025-04-30T06:44:00.059ZDEBGtest_utilization (BackgroundTasks): client response SledAgent = b6d65341-167c-41df-9b5c-41cded99c229 background_task = vpc_route_manager result = Ok(Response { url: "http://[::1]:39365/eip-gateways", status: 204, headers: {"x-request-id": "cffb43fc-c9c4-468b-b86d-ca9627669018", "date": "Wed, 30 Apr 2025 06:44:00 GMT"} })
56092025-04-30T06:44:00.059ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database
56102025-04-30T06:44:00.059ZDEBGtest_utilization (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
56112025-04-30T06:44:00.059ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
56122025-04-30T06:44:00.059ZINFOtest_utilization (dropshot_internal): request completed latency_us = 7423 local_addr = 127.0.0.1:32956 method = GET remote_addr = 127.0.0.1:55797 req_id = 48c95d0c-6981-4e26-9152-d90e27ea3ef0 response_code = 200 uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJhYzU1ZDhkOS1hMjk4LTQ1MzQtODk4Yi1iZWVlMWM5ZTRlNTIifX0%3D
56132025-04-30T06:44:00.060ZDEBGtest_utilization (oximeter): client response result = Ok(Response { url: "http://127.0.0.1:32956/metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJhYzU1ZDhkOS1hMjk4LTQ1MzQtODk4Yi1iZWVlMWM5ZTRlNTIifX0%3D", status: 200, headers: {"content-type": "application/json", "x-request-id": "48c95d0c-6981-4e26-9152-d90e27ea3ef0", "content-length": "29", "date": "Wed, 30 Apr 2025 06:44:00 GMT"} })
56142025-04-30T06:44:00.060ZTRACtest_utilization (oximeter-agent): ignoring request to update existing metric producer, since the endpoint information is the same as the existing collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
56152025-04-30T06:44:00.060ZTRACtest_utilization (oximeter-agent): ignoring request to update existing metric producer, since the endpoint information is the same as the existing collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = ac55d8d9-a298-4534-898b-beee1c9e4e52
56162025-04-30T06:44:00.060ZINFOtest_utilization (oximeter-agent): refreshed list of producers from Nexus collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 file = oximeter/collector/src/agent.rs:637 n_current_tasks = 2 n_pruned_tasks = 0
56172025-04-30T06:44:00.062ZTRACtest_utilization (dropshot_external): loading roles actor = Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. } actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true local_addr = 127.0.0.1:46348 method = POST remote_addr = 127.0.0.1:41283 req_id = bf272e5e-cbe5-41d9-b505-34877ffe12d2 resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet uri = /v1/instances?project=utilization-test-project
56182025-04-30T06:44:00.062ZTRACtest_utilization (dropshot_external): authorize begin action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true local_addr = 127.0.0.1:46348 method = POST remote_addr = 127.0.0.1:41283 req_id = bf272e5e-cbe5-41d9-b505-34877ffe12d2 resource = Database uri = /v1/instances?project=utilization-test-project
56192025-04-30T06:44:00.062ZDEBGtest_utilization (dropshot_external): roles actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true local_addr = 127.0.0.1:46348 method = POST remote_addr = 127.0.0.1:41283 req_id = bf272e5e-cbe5-41d9-b505-34877ffe12d2 roles = RoleSet { roles: {} } uri = /v1/instances?project=utilization-test-project
56202025-04-30T06:44:00.062ZDEBGtest_utilization (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true local_addr = 127.0.0.1:46348 method = POST remote_addr = 127.0.0.1:41283 req_id = bf272e5e-cbe5-41d9-b505-34877ffe12d2 resource = Database result = Ok(()) uri = /v1/instances?project=utilization-test-project
56212025-04-30T06:44:00.066ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database
56222025-04-30T06:44:00.066ZDEBGtest_utilization (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
56232025-04-30T06:44:00.066ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
56242025-04-30T06:44:00.068ZDEBGtest_utilization (dropshot_external): roles actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true local_addr = 127.0.0.1:46348 method = POST remote_addr = 127.0.0.1:41283 req_id = bf272e5e-cbe5-41d9-b505-34877ffe12d2 roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin"), (Silo, 001de000-5110-4000-8000-000000000000, "admin")} } uri = /v1/instances?project=utilization-test-project
56252025-04-30T06:44:00.076ZDEBGtest_utilization (dropshot_external): authorize result action = Read actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true local_addr = 127.0.0.1:46348 method = POST remote_addr = 127.0.0.1:41283 req_id = bf272e5e-cbe5-41d9-b505-34877ffe12d2 resource = Instance { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000000, lookup_type: ById(001de000-5110-4000-8000-000000000000) }, key: 473b39f9-8669-45b3-b3d7-793f7e10fc31, lookup_type: ById(473b39f9-8669-45b3-b3d7-793f7e10fc31) }, key: d7bd2deb-ee27-4661-9e9a-524d734cd6ec, lookup_type: ById(d7bd2deb-ee27-4661-9e9a-524d734cd6ec) } result = Ok(()) uri = /v1/instances?project=utilization-test-project
56262025-04-30T06:44:00.076ZTRACtest_utilization (dropshot_external): authorize begin action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true local_addr = 127.0.0.1:46348 method = POST remote_addr = 127.0.0.1:41283 req_id = bf272e5e-cbe5-41d9-b505-34877ffe12d2 resource = Database uri = /v1/instances?project=utilization-test-project
56272025-04-30T06:44:00.077ZDEBGtest_utilization (dropshot_external): roles actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true local_addr = 127.0.0.1:46348 method = POST remote_addr = 127.0.0.1:41283 req_id = bf272e5e-cbe5-41d9-b505-34877ffe12d2 roles = RoleSet { roles: {} } uri = /v1/instances?project=utilization-test-project
56282025-04-30T06:44:00.077ZDEBGtest_utilization (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true local_addr = 127.0.0.1:46348 method = POST remote_addr = 127.0.0.1:41283 req_id = bf272e5e-cbe5-41d9-b505-34877ffe12d2 resource = Database result = Ok(()) uri = /v1/instances?project=utilization-test-project
56292025-04-30T06:44:00.083ZINFOtest_utilization (dropshot_external): request completed latency_us = 6353612 local_addr = 127.0.0.1:46348 method = POST remote_addr = 127.0.0.1:41283 req_id = bf272e5e-cbe5-41d9-b505-34877ffe12d2 response_code = 201 uri = /v1/instances?project=utilization-test-project
56302025-04-30T06:44:00.086ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database
56312025-04-30T06:44:00.086ZDEBGtest_utilization (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
56322025-04-30T06:44:00.087ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
56332025-04-30T06:44:00.088ZINFOtest_utilization (external client test context): client received response status = 201
56342025-04-30T06:44:00.089ZINFOtest_utilization (external client test context): client request body = Body { inner: BoxBody } method = POST uri = http://127.0.0.1:46348/v1/instances/utilization-test-instance/start?project=utilization-test-project
56352025-04-30T06:44:00.092ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database
56362025-04-30T06:44:00.092ZDEBGtest_utilization (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
56372025-04-30T06:44:00.093ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
56382025-04-30T06:44:00.093ZTRACtest_utilization (dropshot_external): incoming request local_addr = 127.0.0.1:46348 method = POST remote_addr = 127.0.0.1:41283 req_id = 4cf87491-0647-41e3-a483-1824afedf1e7 uri = /v1/instances/utilization-test-instance/start?project=utilization-test-project
56392025-04-30T06:44:00.093ZTRACtest_utilization (dropshot_external): authn: trying SchemeName("spoof") local_addr = 127.0.0.1:46348 method = POST remote_addr = 127.0.0.1:41283 req_id = 4cf87491-0647-41e3-a483-1824afedf1e7 uri = /v1/instances/utilization-test-instance/start?project=utilization-test-project
56402025-04-30T06:44:00.094ZDEBGtest_utilization (dropshot_external): looking up silo for user local_addr = 127.0.0.1:46348 method = POST remote_addr = 127.0.0.1:41283 req_id = 4cf87491-0647-41e3-a483-1824afedf1e7 silo_user_id = 001de000-05e4-4000-8000-000000004007 uri = /v1/instances/utilization-test-instance/start?project=utilization-test-project
56412025-04-30T06:44:00.094ZTRACtest_utilization (ExternalAuthn): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database
56422025-04-30T06:44:00.094ZDEBGtest_utilization (ExternalAuthn): roles roles = RoleSet { roles: {} }
56432025-04-30T06:44:00.094ZDEBGtest_utilization (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
56442025-04-30T06:44:00.097ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database
56452025-04-30T06:44:00.097ZDEBGtest_utilization (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
56462025-04-30T06:44:00.098ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
56472025-04-30T06:44:00.098ZTRACtest_utilization (ExternalAuthn): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database
56482025-04-30T06:44:00.098ZDEBGtest_utilization (ExternalAuthn): roles roles = RoleSet { roles: {} }
56492025-04-30T06:44:00.099ZDEBGtest_utilization (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
56502025-04-30T06:44:00.102ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database
56512025-04-30T06:44:00.102ZDEBGtest_utilization (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
56522025-04-30T06:44:00.103ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
56532025-04-30T06:44:00.103ZTRACtest_utilization (ExternalAuthn): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = SiloUser { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000000, lookup_type: ById(001de000-5110-4000-8000-000000000000) }, key: 001de000-05e4-4000-8000-000000004007, lookup_type: ById(001de000-05e4-4000-8000-000000004007) }
56542025-04-30T06:44:00.103ZTRACtest_utilization (ExternalAuthn): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. } resource_id = 001de000-5110-4000-8000-000000000000 resource_type = Silo
56552025-04-30T06:44:00.103ZTRACtest_utilization (ExternalAuthn): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database
56562025-04-30T06:44:00.103ZDEBGtest_utilization (ExternalAuthn): roles roles = RoleSet { roles: {} }
56572025-04-30T06:44:00.104ZDEBGtest_utilization (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
56582025-04-30T06:44:00.108ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database
56592025-04-30T06:44:00.108ZDEBGtest_utilization (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
56602025-04-30T06:44:00.109ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
56612025-04-30T06:44:00.111ZTRACtest_utilization (ExternalAuthn): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
56622025-04-30T06:44:00.111ZTRACtest_utilization (ExternalAuthn): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database
56632025-04-30T06:44:00.111ZDEBGtest_utilization (ExternalAuthn): roles roles = RoleSet { roles: {} }
56642025-04-30T06:44:00.113ZDEBGtest_utilization (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
56652025-04-30T06:44:00.114ZTRACtest_utilization (BackgroundTasks): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = VpcRouter { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000000, lookup_type: ById(001de000-5110-4000-8000-000000000000) }, key: 473b39f9-8669-45b3-b3d7-793f7e10fc31, lookup_type: ById(473b39f9-8669-45b3-b3d7-793f7e10fc31) }, key: cda41110-774f-439b-b99b-f715ebb74e33, lookup_type: ById(cda41110-774f-439b-b99b-f715ebb74e33) }, key: fd90c8f9-ab47-4a3c-b34d-890ffe0ee4c4, lookup_type: ById(fd90c8f9-ab47-4a3c-b34d-890ffe0ee4c4) }
56662025-04-30T06:44:00.114ZTRACtest_utilization (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = vpc_route_manager resource_id = 473b39f9-8669-45b3-b3d7-793f7e10fc31 resource_type = Project
56672025-04-30T06:44:00.115ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database
56682025-04-30T06:44:00.115ZDEBGtest_utilization (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
56692025-04-30T06:44:00.116ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
56702025-04-30T06:44:00.118ZDEBGtest_utilization (ExternalAuthn): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "external-authenticator")} }
56712025-04-30T06:44:00.121ZDEBGtest_utilization (ExternalAuthn): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = SiloUser { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000000, lookup_type: ById(001de000-5110-4000-8000-000000000000) }, key: 001de000-05e4-4000-8000-000000004007, lookup_type: ById(001de000-05e4-4000-8000-000000004007) } result = Ok(())
56722025-04-30T06:44:00.121ZTRACtest_utilization (ExternalAuthn): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database
56732025-04-30T06:44:00.121ZDEBGtest_utilization (ExternalAuthn): roles roles = RoleSet { roles: {} }
56742025-04-30T06:44:00.122ZDEBGtest_utilization (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
56752025-04-30T06:44:00.122ZTRACtest_utilization (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = vpc_route_manager resource_id = 001de000-5110-4000-8000-000000000000 resource_type = Silo
56762025-04-30T06:44:00.122ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database
56772025-04-30T06:44:00.122ZDEBGtest_utilization (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
56782025-04-30T06:44:00.122ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
56792025-04-30T06:44:00.125ZTRACtest_utilization (ExternalAuthn): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000000, lookup_type: ById(001de000-5110-4000-8000-000000000000) }
56802025-04-30T06:44:00.125ZTRACtest_utilization (ExternalAuthn): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. } resource_id = 001de000-5110-4000-8000-000000000000 resource_type = Silo
56812025-04-30T06:44:00.125ZTRACtest_utilization (ExternalAuthn): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database
56822025-04-30T06:44:00.125ZDEBGtest_utilization (ExternalAuthn): roles roles = RoleSet { roles: {} }
56832025-04-30T06:44:00.125ZDEBGtest_utilization (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
56842025-04-30T06:44:00.128ZTRACtest_utilization (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = vpc_route_manager resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
56852025-04-30T06:44:00.129ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database
56862025-04-30T06:44:00.129ZDEBGtest_utilization (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
56872025-04-30T06:44:00.129ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
56882025-04-30T06:44:00.130ZTRACtest_utilization (ExternalAuthn): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
56892025-04-30T06:44:00.130ZTRACtest_utilization (ExternalAuthn): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database
56902025-04-30T06:44:00.130ZDEBGtest_utilization (ExternalAuthn): roles roles = RoleSet { roles: {} }
56912025-04-30T06:44:00.130ZDEBGtest_utilization (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
56922025-04-30T06:44:00.132ZDEBGtest_utilization (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
56932025-04-30T06:44:00.142ZDEBGtest_utilization (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = VpcRouter { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000000, lookup_type: ById(001de000-5110-4000-8000-000000000000) }, key: 473b39f9-8669-45b3-b3d7-793f7e10fc31, lookup_type: ById(473b39f9-8669-45b3-b3d7-793f7e10fc31) }, key: cda41110-774f-439b-b99b-f715ebb74e33, lookup_type: ById(cda41110-774f-439b-b99b-f715ebb74e33) }, key: fd90c8f9-ab47-4a3c-b34d-890ffe0ee4c4, lookup_type: ById(fd90c8f9-ab47-4a3c-b34d-890ffe0ee4c4) } result = Ok(())
56942025-04-30T06:44:00.142ZTRACtest_utilization (BackgroundTasks): successful lookup of siloed resource "VpcRouter" using built-in user background_task = vpc_route_manager
56952025-04-30T06:44:00.142ZTRACtest_utilization (BackgroundTasks): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = VpcRouter { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000000, lookup_type: ById(001de000-5110-4000-8000-000000000000) }, key: 473b39f9-8669-45b3-b3d7-793f7e10fc31, lookup_type: ById(473b39f9-8669-45b3-b3d7-793f7e10fc31) }, key: cda41110-774f-439b-b99b-f715ebb74e33, lookup_type: ById(cda41110-774f-439b-b99b-f715ebb74e33) }, key: fd90c8f9-ab47-4a3c-b34d-890ffe0ee4c4, lookup_type: ById(fd90c8f9-ab47-4a3c-b34d-890ffe0ee4c4) }
56962025-04-30T06:44:00.142ZTRACtest_utilization (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = vpc_route_manager resource_id = 473b39f9-8669-45b3-b3d7-793f7e10fc31 resource_type = Project
56972025-04-30T06:44:00.142ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database
56982025-04-30T06:44:00.142ZDEBGtest_utilization (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
56992025-04-30T06:44:00.143ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
57002025-04-30T06:44:00.144ZDEBGtest_utilization (ExternalAuthn): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "external-authenticator")} }
57012025-04-30T06:44:00.147ZDEBGtest_utilization (ExternalAuthn): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000000, lookup_type: ById(001de000-5110-4000-8000-000000000000) } result = Ok(())
57022025-04-30T06:44:00.147ZTRACtest_utilization (dropshot_external): authn result: Ok(Context { kind: Authenticated(Details { actor: Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. } }, Some(SiloAuthnPolicy { mapped_fleet_roles: {} })), schemes_tried: [SchemeName("spoof")] }) local_addr = 127.0.0.1:46348 method = POST remote_addr = 127.0.0.1:41283 req_id = 4cf87491-0647-41e3-a483-1824afedf1e7 uri = /v1/instances/utilization-test-instance/start?project=utilization-test-project
57032025-04-30T06:44:00.147ZTRACtest_utilization (dropshot_external): authorize begin action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true local_addr = 127.0.0.1:46348 method = POST remote_addr = 127.0.0.1:41283 req_id = 4cf87491-0647-41e3-a483-1824afedf1e7 resource = Database uri = /v1/instances/utilization-test-instance/start?project=utilization-test-project
57042025-04-30T06:44:00.147ZDEBGtest_utilization (dropshot_external): roles actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true local_addr = 127.0.0.1:46348 method = POST remote_addr = 127.0.0.1:41283 req_id = 4cf87491-0647-41e3-a483-1824afedf1e7 roles = RoleSet { roles: {} } uri = /v1/instances/utilization-test-instance/start?project=utilization-test-project
57052025-04-30T06:44:00.147ZDEBGtest_utilization (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true local_addr = 127.0.0.1:46348 method = POST remote_addr = 127.0.0.1:41283 req_id = 4cf87491-0647-41e3-a483-1824afedf1e7 resource = Database result = Ok(()) uri = /v1/instances/utilization-test-instance/start?project=utilization-test-project
57062025-04-30T06:44:00.149ZTRACtest_utilization (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = vpc_route_manager resource_id = 001de000-5110-4000-8000-000000000000 resource_type = Silo
57072025-04-30T06:44:00.150ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database
57082025-04-30T06:44:00.153ZDEBGtest_utilization (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
57092025-04-30T06:44:00.154ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
57102025-04-30T06:44:00.155ZTRACtest_utilization (dropshot_external): authorize begin action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true local_addr = 127.0.0.1:46348 method = POST remote_addr = 127.0.0.1:41283 req_id = 4cf87491-0647-41e3-a483-1824afedf1e7 resource = Database uri = /v1/instances/utilization-test-instance/start?project=utilization-test-project
57112025-04-30T06:44:00.155ZDEBGtest_utilization (dropshot_external): roles actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true local_addr = 127.0.0.1:46348 method = POST remote_addr = 127.0.0.1:41283 req_id = 4cf87491-0647-41e3-a483-1824afedf1e7 roles = RoleSet { roles: {} } uri = /v1/instances/utilization-test-instance/start?project=utilization-test-project
57122025-04-30T06:44:00.156ZDEBGtest_utilization (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true local_addr = 127.0.0.1:46348 method = POST remote_addr = 127.0.0.1:41283 req_id = 4cf87491-0647-41e3-a483-1824afedf1e7 resource = Database result = Ok(()) uri = /v1/instances/utilization-test-instance/start?project=utilization-test-project
57132025-04-30T06:44:00.157ZTRACtest_utilization (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = vpc_route_manager resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
57142025-04-30T06:44:00.158ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database
57152025-04-30T06:44:00.158ZDEBGtest_utilization (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
57162025-04-30T06:44:00.159ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
57172025-04-30T06:44:00.160ZTRACtest_utilization (dropshot_external): authorize begin action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true local_addr = 127.0.0.1:46348 method = POST remote_addr = 127.0.0.1:41283 req_id = 4cf87491-0647-41e3-a483-1824afedf1e7 resource = Database uri = /v1/instances/utilization-test-instance/start?project=utilization-test-project
57182025-04-30T06:44:00.160ZDEBGtest_utilization (dropshot_external): roles actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true local_addr = 127.0.0.1:46348 method = POST remote_addr = 127.0.0.1:41283 req_id = 4cf87491-0647-41e3-a483-1824afedf1e7 roles = RoleSet { roles: {} } uri = /v1/instances/utilization-test-instance/start?project=utilization-test-project
57192025-04-30T06:44:00.162ZDEBGtest_utilization (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true local_addr = 127.0.0.1:46348 method = POST remote_addr = 127.0.0.1:41283 req_id = 4cf87491-0647-41e3-a483-1824afedf1e7 resource = Database result = Ok(()) uri = /v1/instances/utilization-test-instance/start?project=utilization-test-project
57202025-04-30T06:44:00.163ZDEBGtest_utilization (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
57212025-04-30T06:44:00.173ZDEBGtest_utilization (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = VpcRouter { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000000, lookup_type: ById(001de000-5110-4000-8000-000000000000) }, key: 473b39f9-8669-45b3-b3d7-793f7e10fc31, lookup_type: ById(473b39f9-8669-45b3-b3d7-793f7e10fc31) }, key: cda41110-774f-439b-b99b-f715ebb74e33, lookup_type: ById(cda41110-774f-439b-b99b-f715ebb74e33) }, key: fd90c8f9-ab47-4a3c-b34d-890ffe0ee4c4, lookup_type: ById(fd90c8f9-ab47-4a3c-b34d-890ffe0ee4c4) } result = Ok(())
57222025-04-30T06:44:00.173ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database
57232025-04-30T06:44:00.173ZDEBGtest_utilization (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
57242025-04-30T06:44:00.174ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
57252025-04-30T06:44:00.174ZTRACtest_utilization (dropshot_external): authorize begin action = Modify actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true local_addr = 127.0.0.1:46348 method = POST remote_addr = 127.0.0.1:41283 req_id = 4cf87491-0647-41e3-a483-1824afedf1e7 resource = Instance { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000000, lookup_type: ById(001de000-5110-4000-8000-000000000000) }, key: 473b39f9-8669-45b3-b3d7-793f7e10fc31, lookup_type: ByName("utilization-test-project") }, key: d7bd2deb-ee27-4661-9e9a-524d734cd6ec, lookup_type: ByName("utilization-test-instance") } uri = /v1/instances/utilization-test-instance/start?project=utilization-test-project
57262025-04-30T06:44:00.174ZTRACtest_utilization (dropshot_external): loading roles actor = Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. } actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true local_addr = 127.0.0.1:46348 method = POST remote_addr = 127.0.0.1:41283 req_id = 4cf87491-0647-41e3-a483-1824afedf1e7 resource_id = 473b39f9-8669-45b3-b3d7-793f7e10fc31 resource_type = Project uri = /v1/instances/utilization-test-instance/start?project=utilization-test-project
57272025-04-30T06:44:00.174ZTRACtest_utilization (dropshot_external): authorize begin action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true local_addr = 127.0.0.1:46348 method = POST remote_addr = 127.0.0.1:41283 req_id = 4cf87491-0647-41e3-a483-1824afedf1e7 resource = Database uri = /v1/instances/utilization-test-instance/start?project=utilization-test-project
57282025-04-30T06:44:00.174ZDEBGtest_utilization (dropshot_external): roles actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true local_addr = 127.0.0.1:46348 method = POST remote_addr = 127.0.0.1:41283 req_id = 4cf87491-0647-41e3-a483-1824afedf1e7 roles = RoleSet { roles: {} } uri = /v1/instances/utilization-test-instance/start?project=utilization-test-project
57292025-04-30T06:44:00.175ZDEBGtest_utilization (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true local_addr = 127.0.0.1:46348 method = POST remote_addr = 127.0.0.1:41283 req_id = 4cf87491-0647-41e3-a483-1824afedf1e7 resource = Database result = Ok(()) uri = /v1/instances/utilization-test-instance/start?project=utilization-test-project
57302025-04-30T06:44:00.178ZTRACtest_utilization (dropshot_external): loading roles actor = Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. } actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true local_addr = 127.0.0.1:46348 method = POST remote_addr = 127.0.0.1:41283 req_id = 4cf87491-0647-41e3-a483-1824afedf1e7 resource_id = 001de000-5110-4000-8000-000000000000 resource_type = Silo uri = /v1/instances/utilization-test-instance/start?project=utilization-test-project
57312025-04-30T06:44:00.178ZTRACtest_utilization (dropshot_external): authorize begin action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true local_addr = 127.0.0.1:46348 method = POST remote_addr = 127.0.0.1:41283 req_id = 4cf87491-0647-41e3-a483-1824afedf1e7 resource = Database uri = /v1/instances/utilization-test-instance/start?project=utilization-test-project
57322025-04-30T06:44:00.178ZDEBGtest_utilization (dropshot_external): roles actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true local_addr = 127.0.0.1:46348 method = POST remote_addr = 127.0.0.1:41283 req_id = 4cf87491-0647-41e3-a483-1824afedf1e7 roles = RoleSet { roles: {} } uri = /v1/instances/utilization-test-instance/start?project=utilization-test-project
57332025-04-30T06:44:00.179ZDEBGtest_utilization (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true local_addr = 127.0.0.1:46348 method = POST remote_addr = 127.0.0.1:41283 req_id = 4cf87491-0647-41e3-a483-1824afedf1e7 resource = Database result = Ok(()) uri = /v1/instances/utilization-test-instance/start?project=utilization-test-project
57342025-04-30T06:44:00.180ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database
57352025-04-30T06:44:00.180ZDEBGtest_utilization (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
57362025-04-30T06:44:00.181ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
57372025-04-30T06:44:00.183ZTRACtest_utilization (dropshot_external): loading roles actor = Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. } actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true local_addr = 127.0.0.1:46348 method = POST remote_addr = 127.0.0.1:41283 req_id = 4cf87491-0647-41e3-a483-1824afedf1e7 resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet uri = /v1/instances/utilization-test-instance/start?project=utilization-test-project
57382025-04-30T06:44:00.183ZTRACtest_utilization (dropshot_external): authorize begin action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true local_addr = 127.0.0.1:46348 method = POST remote_addr = 127.0.0.1:41283 req_id = 4cf87491-0647-41e3-a483-1824afedf1e7 resource = Database uri = /v1/instances/utilization-test-instance/start?project=utilization-test-project
57392025-04-30T06:44:00.183ZDEBGtest_utilization (dropshot_external): roles actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true local_addr = 127.0.0.1:46348 method = POST remote_addr = 127.0.0.1:41283 req_id = 4cf87491-0647-41e3-a483-1824afedf1e7 roles = RoleSet { roles: {} } uri = /v1/instances/utilization-test-instance/start?project=utilization-test-project
57402025-04-30T06:44:00.184ZDEBGtest_utilization (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true local_addr = 127.0.0.1:46348 method = POST remote_addr = 127.0.0.1:41283 req_id = 4cf87491-0647-41e3-a483-1824afedf1e7 resource = Database result = Ok(()) uri = /v1/instances/utilization-test-instance/start?project=utilization-test-project
57412025-04-30T06:44:00.187ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database
57422025-04-30T06:44:00.187ZDEBGtest_utilization (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
57432025-04-30T06:44:00.187ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
57442025-04-30T06:44:00.189ZDEBGtest_utilization (dropshot_external): roles actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true local_addr = 127.0.0.1:46348 method = POST remote_addr = 127.0.0.1:41283 req_id = 4cf87491-0647-41e3-a483-1824afedf1e7 roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin"), (Silo, 001de000-5110-4000-8000-000000000000, "admin")} } uri = /v1/instances/utilization-test-instance/start?project=utilization-test-project
57452025-04-30T06:44:00.201ZDEBGtest_utilization (dropshot_external): authorize result action = Modify actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true local_addr = 127.0.0.1:46348 method = POST remote_addr = 127.0.0.1:41283 req_id = 4cf87491-0647-41e3-a483-1824afedf1e7 resource = Instance { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000000, lookup_type: ById(001de000-5110-4000-8000-000000000000) }, key: 473b39f9-8669-45b3-b3d7-793f7e10fc31, lookup_type: ByName("utilization-test-project") }, key: d7bd2deb-ee27-4661-9e9a-524d734cd6ec, lookup_type: ByName("utilization-test-instance") } result = Ok(()) uri = /v1/instances/utilization-test-instance/start?project=utilization-test-project
57462025-04-30T06:44:00.201ZTRACtest_utilization (dropshot_external): authorize begin action = Read actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true local_addr = 127.0.0.1:46348 method = POST remote_addr = 127.0.0.1:41283 req_id = 4cf87491-0647-41e3-a483-1824afedf1e7 resource = Instance { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000000, lookup_type: ById(001de000-5110-4000-8000-000000000000) }, key: 473b39f9-8669-45b3-b3d7-793f7e10fc31, lookup_type: ByName("utilization-test-project") }, key: d7bd2deb-ee27-4661-9e9a-524d734cd6ec, lookup_type: ByName("utilization-test-instance") } uri = /v1/instances/utilization-test-instance/start?project=utilization-test-project
57472025-04-30T06:44:00.201ZTRACtest_utilization (dropshot_external): loading roles actor = Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. } actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true local_addr = 127.0.0.1:46348 method = POST remote_addr = 127.0.0.1:41283 req_id = 4cf87491-0647-41e3-a483-1824afedf1e7 resource_id = 473b39f9-8669-45b3-b3d7-793f7e10fc31 resource_type = Project uri = /v1/instances/utilization-test-instance/start?project=utilization-test-project
57482025-04-30T06:44:00.201ZTRACtest_utilization (dropshot_external): authorize begin action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true local_addr = 127.0.0.1:46348 method = POST remote_addr = 127.0.0.1:41283 req_id = 4cf87491-0647-41e3-a483-1824afedf1e7 resource = Database uri = /v1/instances/utilization-test-instance/start?project=utilization-test-project
57492025-04-30T06:44:00.201ZDEBGtest_utilization (dropshot_external): roles actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true local_addr = 127.0.0.1:46348 method = POST remote_addr = 127.0.0.1:41283 req_id = 4cf87491-0647-41e3-a483-1824afedf1e7 roles = RoleSet { roles: {} } uri = /v1/instances/utilization-test-instance/start?project=utilization-test-project
57502025-04-30T06:44:00.202ZDEBGtest_utilization (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true local_addr = 127.0.0.1:46348 method = POST remote_addr = 127.0.0.1:41283 req_id = 4cf87491-0647-41e3-a483-1824afedf1e7 resource = Database result = Ok(()) uri = /v1/instances/utilization-test-instance/start?project=utilization-test-project
57512025-04-30T06:44:00.202ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database
57522025-04-30T06:44:00.202ZDEBGtest_utilization (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
57532025-04-30T06:44:00.203ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
57542025-04-30T06:44:00.207ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database
57552025-04-30T06:44:00.207ZDEBGtest_utilization (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
57562025-04-30T06:44:00.208ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
57572025-04-30T06:44:00.208ZTRACtest_utilization (dropshot_external): loading roles actor = Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. } actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true local_addr = 127.0.0.1:46348 method = POST remote_addr = 127.0.0.1:41283 req_id = 4cf87491-0647-41e3-a483-1824afedf1e7 resource_id = 001de000-5110-4000-8000-000000000000 resource_type = Silo uri = /v1/instances/utilization-test-instance/start?project=utilization-test-project
57582025-04-30T06:44:00.208ZTRACtest_utilization (dropshot_external): authorize begin action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true local_addr = 127.0.0.1:46348 method = POST remote_addr = 127.0.0.1:41283 req_id = 4cf87491-0647-41e3-a483-1824afedf1e7 resource = Database uri = /v1/instances/utilization-test-instance/start?project=utilization-test-project
57592025-04-30T06:44:00.208ZDEBGtest_utilization (dropshot_external): roles actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true local_addr = 127.0.0.1:46348 method = POST remote_addr = 127.0.0.1:41283 req_id = 4cf87491-0647-41e3-a483-1824afedf1e7 roles = RoleSet { roles: {} } uri = /v1/instances/utilization-test-instance/start?project=utilization-test-project
57602025-04-30T06:44:00.212ZDEBGtest_utilization (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true local_addr = 127.0.0.1:46348 method = POST remote_addr = 127.0.0.1:41283 req_id = 4cf87491-0647-41e3-a483-1824afedf1e7 resource = Database result = Ok(()) uri = /v1/instances/utilization-test-instance/start?project=utilization-test-project
57612025-04-30T06:44:00.227ZTRACtest_utilization (BackgroundTasks): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000000, lookup_type: ById(001de000-5110-4000-8000-000000000000) }, key: 473b39f9-8669-45b3-b3d7-793f7e10fc31, lookup_type: ById(473b39f9-8669-45b3-b3d7-793f7e10fc31) }, key: cda41110-774f-439b-b99b-f715ebb74e33, lookup_type: ById(cda41110-774f-439b-b99b-f715ebb74e33) }, key: be0543da-af63-4252-b6b2-ce2bffd4dd49, lookup_type: ById(be0543da-af63-4252-b6b2-ce2bffd4dd49) }
57622025-04-30T06:44:00.227ZTRACtest_utilization (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = vpc_route_manager resource_id = 473b39f9-8669-45b3-b3d7-793f7e10fc31 resource_type = Project
57632025-04-30T06:44:00.227ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database
57642025-04-30T06:44:00.227ZDEBGtest_utilization (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
57652025-04-30T06:44:00.227ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
57662025-04-30T06:44:00.230ZTRACtest_utilization (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = vpc_route_manager resource_id = 001de000-5110-4000-8000-000000000000 resource_type = Silo
57672025-04-30T06:44:00.232ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database
57682025-04-30T06:44:00.232ZDEBGtest_utilization (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
57692025-04-30T06:44:00.233ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
57702025-04-30T06:44:00.239ZTRACtest_utilization (dropshot_external): loading roles actor = Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. } actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true local_addr = 127.0.0.1:46348 method = POST remote_addr = 127.0.0.1:41283 req_id = 4cf87491-0647-41e3-a483-1824afedf1e7 resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet uri = /v1/instances/utilization-test-instance/start?project=utilization-test-project
57712025-04-30T06:44:00.239ZTRACtest_utilization (dropshot_external): authorize begin action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true local_addr = 127.0.0.1:46348 method = POST remote_addr = 127.0.0.1:41283 req_id = 4cf87491-0647-41e3-a483-1824afedf1e7 resource = Database uri = /v1/instances/utilization-test-instance/start?project=utilization-test-project
57722025-04-30T06:44:00.239ZDEBGtest_utilization (dropshot_external): roles actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true local_addr = 127.0.0.1:46348 method = POST remote_addr = 127.0.0.1:41283 req_id = 4cf87491-0647-41e3-a483-1824afedf1e7 roles = RoleSet { roles: {} } uri = /v1/instances/utilization-test-instance/start?project=utilization-test-project
57732025-04-30T06:44:00.240ZDEBGtest_utilization (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true local_addr = 127.0.0.1:46348 method = POST remote_addr = 127.0.0.1:41283 req_id = 4cf87491-0647-41e3-a483-1824afedf1e7 resource = Database result = Ok(()) uri = /v1/instances/utilization-test-instance/start?project=utilization-test-project
57742025-04-30T06:44:00.241ZTRACtest_utilization (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = vpc_route_manager resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
57752025-04-30T06:44:00.241ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database
57762025-04-30T06:44:00.241ZDEBGtest_utilization (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
57772025-04-30T06:44:00.241ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
57782025-04-30T06:44:00.245ZDEBGtest_utilization (dropshot_external): roles actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true local_addr = 127.0.0.1:46348 method = POST remote_addr = 127.0.0.1:41283 req_id = 4cf87491-0647-41e3-a483-1824afedf1e7 roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin"), (Silo, 001de000-5110-4000-8000-000000000000, "admin")} } uri = /v1/instances/utilization-test-instance/start?project=utilization-test-project
57792025-04-30T06:44:00.259ZDEBGtest_utilization (dropshot_external): authorize result action = Read actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true local_addr = 127.0.0.1:46348 method = POST remote_addr = 127.0.0.1:41283 req_id = 4cf87491-0647-41e3-a483-1824afedf1e7 resource = Instance { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000000, lookup_type: ById(001de000-5110-4000-8000-000000000000) }, key: 473b39f9-8669-45b3-b3d7-793f7e10fc31, lookup_type: ByName("utilization-test-project") }, key: d7bd2deb-ee27-4661-9e9a-524d734cd6ec, lookup_type: ByName("utilization-test-instance") } result = Ok(()) uri = /v1/instances/utilization-test-instance/start?project=utilization-test-project
57802025-04-30T06:44:00.259ZTRACtest_utilization (dropshot_external): authorize begin action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true local_addr = 127.0.0.1:46348 method = POST remote_addr = 127.0.0.1:41283 req_id = 4cf87491-0647-41e3-a483-1824afedf1e7 resource = Database uri = /v1/instances/utilization-test-instance/start?project=utilization-test-project
57812025-04-30T06:44:00.260ZDEBGtest_utilization (dropshot_external): roles actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true local_addr = 127.0.0.1:46348 method = POST remote_addr = 127.0.0.1:41283 req_id = 4cf87491-0647-41e3-a483-1824afedf1e7 roles = RoleSet { roles: {} } uri = /v1/instances/utilization-test-instance/start?project=utilization-test-project
57822025-04-30T06:44:00.262ZDEBGtest_utilization (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true local_addr = 127.0.0.1:46348 method = POST remote_addr = 127.0.0.1:41283 req_id = 4cf87491-0647-41e3-a483-1824afedf1e7 resource = Database result = Ok(()) uri = /v1/instances/utilization-test-instance/start?project=utilization-test-project
57832025-04-30T06:44:00.266ZDEBGtest_utilization (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
57842025-04-30T06:44:00.273ZDEBGtest_utilization (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000000, lookup_type: ById(001de000-5110-4000-8000-000000000000) }, key: 473b39f9-8669-45b3-b3d7-793f7e10fc31, lookup_type: ById(473b39f9-8669-45b3-b3d7-793f7e10fc31) }, key: cda41110-774f-439b-b99b-f715ebb74e33, lookup_type: ById(cda41110-774f-439b-b99b-f715ebb74e33) }, key: be0543da-af63-4252-b6b2-ce2bffd4dd49, lookup_type: ById(be0543da-af63-4252-b6b2-ce2bffd4dd49) } result = Ok(())
57852025-04-30T06:44:00.273ZTRACtest_utilization (BackgroundTasks): successful lookup of siloed resource "VpcSubnet" using built-in user background_task = vpc_route_manager
57862025-04-30T06:44:00.273ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database
57872025-04-30T06:44:00.273ZDEBGtest_utilization (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
57882025-04-30T06:44:00.274ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
57892025-04-30T06:44:00.274ZDEBGtest_utilization (nexus): asked to start an active instance instance_id = d7bd2deb-ee27-4661-9e9a-524d734cd6ec start_reason = User state = Starting
57902025-04-30T06:44:00.274ZINFOtest_utilization (dropshot_external): request completed latency_us = 181279 local_addr = 127.0.0.1:46348 method = POST remote_addr = 127.0.0.1:41283 req_id = 4cf87491-0647-41e3-a483-1824afedf1e7 response_code = 202 uri = /v1/instances/utilization-test-instance/start?project=utilization-test-project
57912025-04-30T06:44:00.275ZINFOtest_utilization (external client test context): client received response status = 202
57922025-04-30T06:44:00.275ZINFOtest_utilization (external client test context): client request body = Body { inner: BoxBody } method = GET uri = http://127.0.0.1:46348/v1/system/utilization/silos/default-silo
57932025-04-30T06:44:00.276ZTRACtest_utilization (dropshot_external): incoming request local_addr = 127.0.0.1:46348 method = GET remote_addr = 127.0.0.1:41283 req_id = eb590556-edc6-4009-9858-248d6007c68b uri = /v1/system/utilization/silos/default-silo
57942025-04-30T06:44:00.276ZTRACtest_utilization (dropshot_external): authn: trying SchemeName("spoof") local_addr = 127.0.0.1:46348 method = GET remote_addr = 127.0.0.1:41283 req_id = eb590556-edc6-4009-9858-248d6007c68b uri = /v1/system/utilization/silos/default-silo
57952025-04-30T06:44:00.276ZDEBGtest_utilization (dropshot_external): looking up silo for user local_addr = 127.0.0.1:46348 method = GET remote_addr = 127.0.0.1:41283 req_id = eb590556-edc6-4009-9858-248d6007c68b silo_user_id = 001de000-05e4-4000-8000-000000004007 uri = /v1/system/utilization/silos/default-silo
57962025-04-30T06:44:00.276ZTRACtest_utilization (ExternalAuthn): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database
57972025-04-30T06:44:00.276ZDEBGtest_utilization (ExternalAuthn): roles roles = RoleSet { roles: {} }
57982025-04-30T06:44:00.277ZDEBGtest_utilization (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
57992025-04-30T06:44:00.278ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database
58002025-04-30T06:44:00.278ZDEBGtest_utilization (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
58012025-04-30T06:44:00.278ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
58022025-04-30T06:44:00.280ZTRACtest_utilization (ExternalAuthn): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database
58032025-04-30T06:44:00.280ZDEBGtest_utilization (ExternalAuthn): roles roles = RoleSet { roles: {} }
58042025-04-30T06:44:00.281ZDEBGtest_utilization (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
58052025-04-30T06:44:00.282ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database
58062025-04-30T06:44:00.282ZDEBGtest_utilization (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
58072025-04-30T06:44:00.283ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
58082025-04-30T06:44:00.285ZTRACtest_utilization (ExternalAuthn): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = SiloUser { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000000, lookup_type: ById(001de000-5110-4000-8000-000000000000) }, key: 001de000-05e4-4000-8000-000000004007, lookup_type: ById(001de000-05e4-4000-8000-000000004007) }
58092025-04-30T06:44:00.286ZTRACtest_utilization (ExternalAuthn): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. } resource_id = 001de000-5110-4000-8000-000000000000 resource_type = Silo
58102025-04-30T06:44:00.286ZTRACtest_utilization (ExternalAuthn): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database
58112025-04-30T06:44:00.286ZDEBGtest_utilization (ExternalAuthn): roles roles = RoleSet { roles: {} }
58122025-04-30T06:44:00.288ZDEBGtest_utilization (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
58132025-04-30T06:44:00.288ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database
58142025-04-30T06:44:00.288ZDEBGtest_utilization (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
58152025-04-30T06:44:00.289ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
58162025-04-30T06:44:00.290ZTRACtest_utilization (ExternalAuthn): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
58172025-04-30T06:44:00.291ZTRACtest_utilization (ExternalAuthn): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database
58182025-04-30T06:44:00.291ZDEBGtest_utilization (ExternalAuthn): roles roles = RoleSet { roles: {} }
58192025-04-30T06:44:00.292ZDEBGtest_utilization (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
58202025-04-30T06:44:00.293ZTRACtest_utilization (BackgroundTasks): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000000, lookup_type: ById(001de000-5110-4000-8000-000000000000) }, key: 473b39f9-8669-45b3-b3d7-793f7e10fc31, lookup_type: ById(473b39f9-8669-45b3-b3d7-793f7e10fc31) }, key: cda41110-774f-439b-b99b-f715ebb74e33, lookup_type: ById(cda41110-774f-439b-b99b-f715ebb74e33) }, key: be0543da-af63-4252-b6b2-ce2bffd4dd49, lookup_type: ByName("default") }
58212025-04-30T06:44:00.293ZTRACtest_utilization (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = vpc_route_manager resource_id = 473b39f9-8669-45b3-b3d7-793f7e10fc31 resource_type = Project
58222025-04-30T06:44:00.293ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database
58232025-04-30T06:44:00.293ZDEBGtest_utilization (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
58242025-04-30T06:44:00.293ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
58252025-04-30T06:44:00.295ZDEBGtest_utilization (ExternalAuthn): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "external-authenticator")} }
58262025-04-30T06:44:00.301ZDEBGtest_utilization (ExternalAuthn): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = SiloUser { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000000, lookup_type: ById(001de000-5110-4000-8000-000000000000) }, key: 001de000-05e4-4000-8000-000000004007, lookup_type: ById(001de000-05e4-4000-8000-000000004007) } result = Ok(())
58272025-04-30T06:44:00.301ZTRACtest_utilization (ExternalAuthn): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database
58282025-04-30T06:44:00.302ZDEBGtest_utilization (ExternalAuthn): roles roles = RoleSet { roles: {} }
58292025-04-30T06:44:00.304ZDEBGtest_utilization (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
58302025-04-30T06:44:00.305ZTRACtest_utilization (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = vpc_route_manager resource_id = 001de000-5110-4000-8000-000000000000 resource_type = Silo
58312025-04-30T06:44:00.305ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database
58322025-04-30T06:44:00.305ZDEBGtest_utilization (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
58332025-04-30T06:44:00.305ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
58342025-04-30T06:44:00.307ZTRACtest_utilization (ExternalAuthn): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000000, lookup_type: ById(001de000-5110-4000-8000-000000000000) }
58352025-04-30T06:44:00.307ZTRACtest_utilization (ExternalAuthn): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. } resource_id = 001de000-5110-4000-8000-000000000000 resource_type = Silo
58362025-04-30T06:44:00.307ZTRACtest_utilization (ExternalAuthn): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database
58372025-04-30T06:44:00.310ZDEBGtest_utilization (ExternalAuthn): roles roles = RoleSet { roles: {} }
58382025-04-30T06:44:00.310ZDEBGtest_utilization (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
58392025-04-30T06:44:00.312ZTRACtest_utilization (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = vpc_route_manager resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
58402025-04-30T06:44:00.312ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database
58412025-04-30T06:44:00.312ZDEBGtest_utilization (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
58422025-04-30T06:44:00.313ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
58432025-04-30T06:44:00.313ZTRACtest_utilization (ExternalAuthn): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
58442025-04-30T06:44:00.314ZTRACtest_utilization (ExternalAuthn): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database
58452025-04-30T06:44:00.314ZDEBGtest_utilization (ExternalAuthn): roles roles = RoleSet { roles: {} }
58462025-04-30T06:44:00.314ZDEBGtest_utilization (ExternalAuthn): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Database result = Ok(())
58472025-04-30T06:44:00.319ZDEBGtest_utilization (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
58482025-04-30T06:44:00.339ZDEBGtest_utilization (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000000, lookup_type: ById(001de000-5110-4000-8000-000000000000) }, key: 473b39f9-8669-45b3-b3d7-793f7e10fc31, lookup_type: ById(473b39f9-8669-45b3-b3d7-793f7e10fc31) }, key: cda41110-774f-439b-b99b-f715ebb74e33, lookup_type: ById(cda41110-774f-439b-b99b-f715ebb74e33) }, key: be0543da-af63-4252-b6b2-ce2bffd4dd49, lookup_type: ByName("default") } result = Ok(())
58492025-04-30T06:44:00.339ZTRACtest_utilization (BackgroundTasks): successful lookup of siloed resource "VpcSubnet" using built-in user background_task = vpc_route_manager
58502025-04-30T06:44:00.339ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database
58512025-04-30T06:44:00.339ZDEBGtest_utilization (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
58522025-04-30T06:44:00.340ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
58532025-04-30T06:44:00.340ZDEBGtest_utilization (ExternalAuthn): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "external-authenticator")} }
58542025-04-30T06:44:00.345ZDEBGtest_utilization (ExternalAuthn): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000003, .. }) resource = Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000000, lookup_type: ById(001de000-5110-4000-8000-000000000000) } result = Ok(())
58552025-04-30T06:44:00.345ZTRACtest_utilization (dropshot_external): authn result: Ok(Context { kind: Authenticated(Details { actor: Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. } }, Some(SiloAuthnPolicy { mapped_fleet_roles: {} })), schemes_tried: [SchemeName("spoof")] }) local_addr = 127.0.0.1:46348 method = GET remote_addr = 127.0.0.1:41283 req_id = eb590556-edc6-4009-9858-248d6007c68b uri = /v1/system/utilization/silos/default-silo
58562025-04-30T06:44:00.345ZTRACtest_utilization (dropshot_external): authorize begin action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true local_addr = 127.0.0.1:46348 method = GET remote_addr = 127.0.0.1:41283 req_id = eb590556-edc6-4009-9858-248d6007c68b resource = Database uri = /v1/system/utilization/silos/default-silo
58572025-04-30T06:44:00.346ZDEBGtest_utilization (dropshot_external): roles actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true local_addr = 127.0.0.1:46348 method = GET remote_addr = 127.0.0.1:41283 req_id = eb590556-edc6-4009-9858-248d6007c68b roles = RoleSet { roles: {} } uri = /v1/system/utilization/silos/default-silo
58582025-04-30T06:44:00.347ZDEBGtest_utilization (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true local_addr = 127.0.0.1:46348 method = GET remote_addr = 127.0.0.1:41283 req_id = eb590556-edc6-4009-9858-248d6007c68b resource = Database result = Ok(()) uri = /v1/system/utilization/silos/default-silo
58592025-04-30T06:44:00.353ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database
58602025-04-30T06:44:00.353ZDEBGtest_utilization (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
58612025-04-30T06:44:00.353ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
58622025-04-30T06:44:00.354ZTRACtest_utilization (dropshot_external): authorize begin action = Read actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true local_addr = 127.0.0.1:46348 method = GET remote_addr = 127.0.0.1:41283 req_id = eb590556-edc6-4009-9858-248d6007c68b resource = Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000000, lookup_type: ByName("default-silo") } uri = /v1/system/utilization/silos/default-silo
58632025-04-30T06:44:00.354ZTRACtest_utilization (dropshot_external): loading roles actor = Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. } actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true local_addr = 127.0.0.1:46348 method = GET remote_addr = 127.0.0.1:41283 req_id = eb590556-edc6-4009-9858-248d6007c68b resource_id = 001de000-5110-4000-8000-000000000000 resource_type = Silo uri = /v1/system/utilization/silos/default-silo
58642025-04-30T06:44:00.354ZTRACtest_utilization (dropshot_external): authorize begin action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true local_addr = 127.0.0.1:46348 method = GET remote_addr = 127.0.0.1:41283 req_id = eb590556-edc6-4009-9858-248d6007c68b resource = Database uri = /v1/system/utilization/silos/default-silo
58652025-04-30T06:44:00.354ZDEBGtest_utilization (dropshot_external): roles actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true local_addr = 127.0.0.1:46348 method = GET remote_addr = 127.0.0.1:41283 req_id = eb590556-edc6-4009-9858-248d6007c68b roles = RoleSet { roles: {} } uri = /v1/system/utilization/silos/default-silo
58662025-04-30T06:44:00.354ZDEBGtest_utilization (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true local_addr = 127.0.0.1:46348 method = GET remote_addr = 127.0.0.1:41283 req_id = eb590556-edc6-4009-9858-248d6007c68b resource = Database result = Ok(()) uri = /v1/system/utilization/silos/default-silo
58672025-04-30T06:44:00.356ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database
58682025-04-30T06:44:00.356ZDEBGtest_utilization (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
58692025-04-30T06:44:00.357ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
58702025-04-30T06:44:00.358ZDEBGtest_utilization (results-sink): inserting 1 samples into database collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
58712025-04-30T06:44:00.358ZDEBGtest_utilization (results-sink-cluster): inserting 1 samples into database collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
58722025-04-30T06:44:00.358ZDEBGtest_utilization (clickhouse-client): unrolling 1 total samples collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 4d1485f0-110e-4161-9a33-c4610d68be5d
58732025-04-30T06:44:00.359ZTRACtest_utilization (clickhouse-client): inserting data collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 4d1485f0-110e-4161-9a33-c4610d68be5d n_columns = 4 n_rows = 2 sql = INSERT INTO oximeter.fields_string FORMAT Native
58742025-04-30T06:44:00.360ZTRACtest_utilization (dropshot_external): loading roles actor = Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. } actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true local_addr = 127.0.0.1:46348 method = GET remote_addr = 127.0.0.1:41283 req_id = eb590556-edc6-4009-9858-248d6007c68b resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet uri = /v1/system/utilization/silos/default-silo
58752025-04-30T06:44:00.360ZTRACtest_utilization (dropshot_external): authorize begin action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true local_addr = 127.0.0.1:46348 method = GET remote_addr = 127.0.0.1:41283 req_id = eb590556-edc6-4009-9858-248d6007c68b resource = Database uri = /v1/system/utilization/silos/default-silo
58762025-04-30T06:44:00.360ZDEBGtest_utilization (dropshot_external): roles actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true local_addr = 127.0.0.1:46348 method = GET remote_addr = 127.0.0.1:41283 req_id = eb590556-edc6-4009-9858-248d6007c68b roles = RoleSet { roles: {} } uri = /v1/system/utilization/silos/default-silo
58772025-04-30T06:44:00.360ZDEBGtest_utilization (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true local_addr = 127.0.0.1:46348 method = GET remote_addr = 127.0.0.1:41283 req_id = eb590556-edc6-4009-9858-248d6007c68b resource = Database result = Ok(()) uri = /v1/system/utilization/silos/default-silo
58782025-04-30T06:44:00.362ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database
58792025-04-30T06:44:00.363ZDEBGtest_utilization (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
58802025-04-30T06:44:00.364ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
58812025-04-30T06:44:00.365ZDEBGtest_utilization (clickhouse-client): inserted rows into table collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 4d1485f0-110e-4161-9a33-c4610d68be5d n_rows = 2 table_name = fields_string
58822025-04-30T06:44:00.365ZTRACtest_utilization (clickhouse-client): inserting data collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 4d1485f0-110e-4161-9a33-c4610d68be5d n_columns = 4 n_rows = 1 sql = INSERT INTO oximeter.measurements_i64 FORMAT Native
58832025-04-30T06:44:00.365ZDEBGtest_utilization (dropshot_external): roles actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true local_addr = 127.0.0.1:46348 method = GET remote_addr = 127.0.0.1:41283 req_id = eb590556-edc6-4009-9858-248d6007c68b roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin"), (Silo, 001de000-5110-4000-8000-000000000000, "admin")} } uri = /v1/system/utilization/silos/default-silo
58842025-04-30T06:44:00.368ZDEBGtest_utilization (dropshot_external): authorize result action = Read actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true local_addr = 127.0.0.1:46348 method = GET remote_addr = 127.0.0.1:41283 req_id = eb590556-edc6-4009-9858-248d6007c68b resource = Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000000, lookup_type: ByName("default-silo") } result = Ok(()) uri = /v1/system/utilization/silos/default-silo
58852025-04-30T06:44:00.368ZTRACtest_utilization (dropshot_external): authorize begin action = Read actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true local_addr = 127.0.0.1:46348 method = GET remote_addr = 127.0.0.1:41283 req_id = eb590556-edc6-4009-9858-248d6007c68b resource = Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000000, lookup_type: ByName("default-silo") } uri = /v1/system/utilization/silos/default-silo
58862025-04-30T06:44:00.368ZTRACtest_utilization (dropshot_external): loading roles actor = Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. } actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true local_addr = 127.0.0.1:46348 method = GET remote_addr = 127.0.0.1:41283 req_id = eb590556-edc6-4009-9858-248d6007c68b resource_id = 001de000-5110-4000-8000-000000000000 resource_type = Silo uri = /v1/system/utilization/silos/default-silo
58872025-04-30T06:44:00.368ZTRACtest_utilization (dropshot_external): authorize begin action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true local_addr = 127.0.0.1:46348 method = GET remote_addr = 127.0.0.1:41283 req_id = eb590556-edc6-4009-9858-248d6007c68b resource = Database uri = /v1/system/utilization/silos/default-silo
58882025-04-30T06:44:00.368ZDEBGtest_utilization (dropshot_external): roles actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true local_addr = 127.0.0.1:46348 method = GET remote_addr = 127.0.0.1:41283 req_id = eb590556-edc6-4009-9858-248d6007c68b roles = RoleSet { roles: {} } uri = /v1/system/utilization/silos/default-silo
58892025-04-30T06:44:00.369ZDEBGtest_utilization (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true local_addr = 127.0.0.1:46348 method = GET remote_addr = 127.0.0.1:41283 req_id = eb590556-edc6-4009-9858-248d6007c68b resource = Database result = Ok(()) uri = /v1/system/utilization/silos/default-silo
58902025-04-30T06:44:00.372ZTRACtest_utilization (BackgroundTasks): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = InternetGateway { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000000, lookup_type: ById(001de000-5110-4000-8000-000000000000) }, key: 473b39f9-8669-45b3-b3d7-793f7e10fc31, lookup_type: ById(473b39f9-8669-45b3-b3d7-793f7e10fc31) }, key: cda41110-774f-439b-b99b-f715ebb74e33, lookup_type: ById(cda41110-774f-439b-b99b-f715ebb74e33) }, key: 5567c57c-3b46-4ca0-968f-03632bf48df0, lookup_type: ByName("default") }
58912025-04-30T06:44:00.372ZTRACtest_utilization (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = vpc_route_manager resource_id = 473b39f9-8669-45b3-b3d7-793f7e10fc31 resource_type = Project
58922025-04-30T06:44:00.372ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database
58932025-04-30T06:44:00.372ZDEBGtest_utilization (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
58942025-04-30T06:44:00.373ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
58952025-04-30T06:44:00.374ZTRACtest_utilization (dropshot_external): loading roles actor = Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. } actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true local_addr = 127.0.0.1:46348 method = GET remote_addr = 127.0.0.1:41283 req_id = eb590556-edc6-4009-9858-248d6007c68b resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet uri = /v1/system/utilization/silos/default-silo
58962025-04-30T06:44:00.374ZTRACtest_utilization (dropshot_external): authorize begin action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true local_addr = 127.0.0.1:46348 method = GET remote_addr = 127.0.0.1:41283 req_id = eb590556-edc6-4009-9858-248d6007c68b resource = Database uri = /v1/system/utilization/silos/default-silo
58972025-04-30T06:44:00.374ZDEBGtest_utilization (dropshot_external): roles actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true local_addr = 127.0.0.1:46348 method = GET remote_addr = 127.0.0.1:41283 req_id = eb590556-edc6-4009-9858-248d6007c68b roles = RoleSet { roles: {} } uri = /v1/system/utilization/silos/default-silo
58982025-04-30T06:44:00.375ZDEBGtest_utilization (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true local_addr = 127.0.0.1:46348 method = GET remote_addr = 127.0.0.1:41283 req_id = eb590556-edc6-4009-9858-248d6007c68b resource = Database result = Ok(()) uri = /v1/system/utilization/silos/default-silo
58992025-04-30T06:44:00.376ZDEBGtest_utilization (clickhouse-client): inserted rows into table collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 4d1485f0-110e-4161-9a33-c4610d68be5d n_rows = 1 table_name = measurements_i64
59002025-04-30T06:44:00.376ZTRACtest_utilization (results-sink): successfully inserted samples collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
59012025-04-30T06:44:00.376ZTRACtest_utilization (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = vpc_route_manager resource_id = 001de000-5110-4000-8000-000000000000 resource_type = Silo
59022025-04-30T06:44:00.376ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database
59032025-04-30T06:44:00.376ZDEBGtest_utilization (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
59042025-04-30T06:44:00.378ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
59052025-04-30T06:44:00.379ZDEBGtest_utilization (dropshot_external): roles actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true local_addr = 127.0.0.1:46348 method = GET remote_addr = 127.0.0.1:41283 req_id = eb590556-edc6-4009-9858-248d6007c68b roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin"), (Silo, 001de000-5110-4000-8000-000000000000, "admin")} } uri = /v1/system/utilization/silos/default-silo
59062025-04-30T06:44:00.381ZDEBGtest_utilization (dropshot_external): authorize result action = Read actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true local_addr = 127.0.0.1:46348 method = GET remote_addr = 127.0.0.1:41283 req_id = eb590556-edc6-4009-9858-248d6007c68b resource = Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000000, lookup_type: ByName("default-silo") } result = Ok(()) uri = /v1/system/utilization/silos/default-silo
59072025-04-30T06:44:00.381ZTRACtest_utilization (dropshot_external): authorize begin action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true local_addr = 127.0.0.1:46348 method = GET remote_addr = 127.0.0.1:41283 req_id = eb590556-edc6-4009-9858-248d6007c68b resource = Database uri = /v1/system/utilization/silos/default-silo
59082025-04-30T06:44:00.381ZDEBGtest_utilization (dropshot_external): roles actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true local_addr = 127.0.0.1:46348 method = GET remote_addr = 127.0.0.1:41283 req_id = eb590556-edc6-4009-9858-248d6007c68b roles = RoleSet { roles: {} } uri = /v1/system/utilization/silos/default-silo
59092025-04-30T06:44:00.382ZDEBGtest_utilization (dropshot_external): authorize result action = Query actor = Some(Actor::SiloUser { silo_user_id: 001de000-05e4-4000-8000-000000004007, silo_id: 001de000-5110-4000-8000-000000000000, .. }) actor_id = 001de000-05e4-4000-8000-000000004007 authenticated = true local_addr = 127.0.0.1:46348 method = GET remote_addr = 127.0.0.1:41283 req_id = eb590556-edc6-4009-9858-248d6007c68b resource = Database result = Ok(()) uri = /v1/system/utilization/silos/default-silo
59102025-04-30T06:44:00.383ZTRACtest_utilization (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = vpc_route_manager resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
59112025-04-30T06:44:00.383ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database
59122025-04-30T06:44:00.383ZDEBGtest_utilization (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
59132025-04-30T06:44:00.384ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
59142025-04-30T06:44:00.388ZDEBGtest_utilization (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
59152025-04-30T06:44:00.397ZDEBGtest_utilization (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = InternetGateway { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000000, lookup_type: ById(001de000-5110-4000-8000-000000000000) }, key: 473b39f9-8669-45b3-b3d7-793f7e10fc31, lookup_type: ById(473b39f9-8669-45b3-b3d7-793f7e10fc31) }, key: cda41110-774f-439b-b99b-f715ebb74e33, lookup_type: ById(cda41110-774f-439b-b99b-f715ebb74e33) }, key: 5567c57c-3b46-4ca0-968f-03632bf48df0, lookup_type: ByName("default") } result = Ok(())
59162025-04-30T06:44:00.397ZTRACtest_utilization (BackgroundTasks): successful lookup of siloed resource "InternetGateway" using built-in user background_task = vpc_route_manager
59172025-04-30T06:44:00.398ZDEBGtest_utilization (BackgroundTasks): client request SledAgent = b6d65341-167c-41df-9b5c-41cded99c229 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[::1]:39365/vpc-routes
59182025-04-30T06:44:00.398ZINFOtest_utilization (dropshot_external): request completed latency_us = 121984 local_addr = 127.0.0.1:46348 method = GET remote_addr = 127.0.0.1:41283 req_id = eb590556-edc6-4009-9858-248d6007c68b response_code = 200 uri = /v1/system/utilization/silos/default-silo
59192025-04-30T06:44:00.399ZTRACtest_utilization (dropshot): incoming request local_addr = [::1]:39365 method = PUT remote_addr = [::1]:33244 req_id = 3bce63c9-9fd9-480c-bc11-f9da55cd9565 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /vpc-routes
59202025-04-30T06:44:00.400ZINFOtest_utilization (dropshot): request completed latency_us = 1433 local_addr = [::1]:39365 method = PUT remote_addr = [::1]:33244 req_id = 3bce63c9-9fd9-480c-bc11-f9da55cd9565 response_code = 204 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /vpc-routes
59212025-04-30T06:44:00.400ZINFOtest_utilization (external client test context): client received response status = 200
59222025-04-30T06:44:00.402ZDEBGtest_utilization (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:36604 method = GET path = /crucible/0/regions server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
59232025-04-30T06:44:00.402ZDEBGtest_utilization (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:36604 method = POST path = /crucible/0/regions server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
59242025-04-30T06:44:00.402ZDEBGtest_utilization (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:36604 method = DELETE path = /crucible/0/regions/{id} server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
59252025-04-30T06:44:00.402ZDEBGtest_utilization (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:36604 method = GET path = /crucible/0/regions/{id} server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
59262025-04-30T06:44:00.402ZDEBGtest_utilization (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:36604 method = GET path = /crucible/0/regions/{id}/snapshots server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
59272025-04-30T06:44:00.402ZDEBGtest_utilization (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:36604 method = DELETE path = /crucible/0/regions/{id}/snapshots/{name} server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
59282025-04-30T06:44:00.402ZDEBGtest_utilization (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:36604 method = GET path = /crucible/0/regions/{id}/snapshots/{name} server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
59292025-04-30T06:44:00.402ZDEBGtest_utilization (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:36604 method = DELETE path = /crucible/0/regions/{id}/snapshots/{name}/run server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
59302025-04-30T06:44:00.402ZDEBGtest_utilization (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:36604 method = POST path = /crucible/0/regions/{id}/snapshots/{name}/run server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
59312025-04-30T06:44:00.402ZINFOtest_utilization (Simulated CrucibleAgent Dropshot Server): listening kind = storage local_addr = [::1]:36604 server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
59322025-04-30T06:44:00.403ZDEBGtest_utilization (Simulated CrucibleAgent Dropshot Server): successfully registered DTrace USDT probes kind = storage local_addr = [::1]:36604 server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
59332025-04-30T06:44:00.403ZINFOtest_utilization (SledAgent): Created Simulated Crucible Server address = [::1]:36604 kind = storage server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
59342025-04-30T06:44:00.403ZINFOtest_utilization (nexus): inserting test physical disk model = test-model physical_disk_id = 5d34b1ef-1afe-4ba2-b042-2f318e4fd7a7 serial = totally-unique-serial: 5d34b1ef-1afe-4ba2-b042-2f318e4fd7a7 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 vendor = test-vendor
59352025-04-30T06:44:00.403ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
59362025-04-30T06:44:00.403ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
59372025-04-30T06:44:00.404ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
59382025-04-30T06:44:00.404ZDEBGtest_utilization (BackgroundTasks): client response SledAgent = b6d65341-167c-41df-9b5c-41cded99c229 background_task = vpc_route_manager result = Ok(Response { url: "http://[::1]:39365/vpc-routes", status: 204, headers: {"x-request-id": "3bce63c9-9fd9-480c-bc11-f9da55cd9565", "date": "Wed, 30 Apr 2025 06:44:00 GMT"} })
59392025-04-30T06:44:00.405ZDEBGtest_utilization (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 552.706192ms iteration = 2 status = {}
59402025-04-30T06:44:00.415ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
59412025-04-30T06:44:00.415ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
59422025-04-30T06:44:00.416ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
59432025-04-30T06:44:00.420ZTRACtest_utilization (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Fleet
59442025-04-30T06:44:00.420ZTRACtest_utilization (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
59452025-04-30T06:44:00.421ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
59462025-04-30T06:44:00.421ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
59472025-04-30T06:44:00.421ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
59482025-04-30T06:44:00.423ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
59492025-04-30T06:44:00.429ZDEBGtest_utilization (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Fleet result = Ok(())
59502025-04-30T06:44:00.459ZWARNtest_utilization (results-sink-cluster): failed to insert some results into metric DB collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 error = Failed to check out connection to database: No backends found for this service file = oximeter/collector/src/results_sink.rs:92
59512025-04-30T06:44:00.597ZINFOtest_utilization (nexus): upserting zpool physical_disk_id = 5d34b1ef-1afe-4ba2-b042-2f318e4fd7a7 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 zpool_id = 309f6563-9474-4815-b285-47e0d4f66174
59522025-04-30T06:44:00.597ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
59532025-04-30T06:44:00.597ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
59542025-04-30T06:44:00.598ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
59552025-04-30T06:44:00.599ZTRACtest_utilization (InternalApi): authorize begin action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Fleet
59562025-04-30T06:44:00.599ZTRACtest_utilization (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
59572025-04-30T06:44:00.599ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
59582025-04-30T06:44:00.599ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
59592025-04-30T06:44:00.600ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
59602025-04-30T06:44:00.602ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
59612025-04-30T06:44:00.604ZDEBGtest_utilization (InternalApi): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Fleet result = Ok(())
59622025-04-30T06:44:00.614ZINFOtest_utilization (nexus): upserting Crucible dataset dataset_id = 8c8262d1-c1e1-4c59-b29e-03ed0f3c05b4 zpool_id = 309f6563-9474-4815-b285-47e0d4f66174
59632025-04-30T06:44:00.622ZINFOtest_utilization (nexus): inserting test physical disk model = test-model physical_disk_id = 5d34b1ef-1afe-4ba2-b042-2f318e4fd7a7 serial = totally-unique-serial: 5d34b1ef-1afe-4ba2-b042-2f318e4fd7a7 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 vendor = test-vendor
59642025-04-30T06:44:00.622ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
59652025-04-30T06:44:00.622ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
59662025-04-30T06:44:00.623ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
59672025-04-30T06:44:00.625ZTRACtest_utilization (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = PhysicalDisk { parent: Fleet, key: 5d34b1ef-1afe-4ba2-b042-2f318e4fd7a7 (physical_disk), lookup_type: ById(5d34b1ef-1afe-4ba2-b042-2f318e4fd7a7) }
59682025-04-30T06:44:00.625ZTRACtest_utilization (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
59692025-04-30T06:44:00.625ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
59702025-04-30T06:44:00.626ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
59712025-04-30T06:44:00.626ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
59722025-04-30T06:44:00.628ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
59732025-04-30T06:44:00.633ZDEBGtest_utilization (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = PhysicalDisk { parent: Fleet, key: 5d34b1ef-1afe-4ba2-b042-2f318e4fd7a7 (physical_disk), lookup_type: ById(5d34b1ef-1afe-4ba2-b042-2f318e4fd7a7) } result = Ok(())
59742025-04-30T06:44:00.633ZINFOtest_utilization (nexus): upserting zpool physical_disk_id = 5d34b1ef-1afe-4ba2-b042-2f318e4fd7a7 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 zpool_id = 309f6563-9474-4815-b285-47e0d4f66174
59752025-04-30T06:44:00.633ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
59762025-04-30T06:44:00.633ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
59772025-04-30T06:44:00.633ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
59782025-04-30T06:44:00.635ZTRACtest_utilization (InternalApi): authorize begin action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Fleet
59792025-04-30T06:44:00.635ZTRACtest_utilization (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
59802025-04-30T06:44:00.635ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
59812025-04-30T06:44:00.635ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
59822025-04-30T06:44:00.635ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
59832025-04-30T06:44:00.638ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
59842025-04-30T06:44:00.641ZDEBGtest_utilization (InternalApi): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Fleet result = Ok(())
59852025-04-30T06:44:00.651ZTRACtest_utilization (InternalApi): authorize begin action = CreateChild actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Fleet
59862025-04-30T06:44:00.652ZTRACtest_utilization (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
59872025-04-30T06:44:00.653ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
59882025-04-30T06:44:00.653ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
59892025-04-30T06:44:00.653ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
59902025-04-30T06:44:00.657ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
59912025-04-30T06:44:00.666ZDEBGtest_utilization (InternalApi): authorize result action = CreateChild actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Fleet result = Ok(())
59922025-04-30T06:44:00.666ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
59932025-04-30T06:44:00.666ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
59942025-04-30T06:44:00.667ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
59952025-04-30T06:44:00.674ZTRACtest_utilization (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory
59962025-04-30T06:44:00.674ZTRACtest_utilization (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
59972025-04-30T06:44:00.674ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
59982025-04-30T06:44:00.674ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
59992025-04-30T06:44:00.674ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
60002025-04-30T06:44:00.674ZDEBGtest_utilization (BackgroundTasks): activating background_task = inventory_collection iteration = 4 reason = Signaled
60012025-04-30T06:44:00.674ZTRACtest_utilization (BackgroundTasks): authorize begin action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory
60022025-04-30T06:44:00.674ZTRACtest_utilization (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
60032025-04-30T06:44:00.675ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
60042025-04-30T06:44:00.675ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
60052025-04-30T06:44:00.675ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
60062025-04-30T06:44:00.677ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
60072025-04-30T06:44:00.683ZDEBGtest_utilization (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory result = Ok(())
60082025-04-30T06:44:00.683ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
60092025-04-30T06:44:00.683ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
60102025-04-30T06:44:00.683ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
60112025-04-30T06:44:00.683ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
60122025-04-30T06:44:00.687ZDEBGtest_utilization (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory result = Ok(())
60132025-04-30T06:44:00.687ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
60142025-04-30T06:44:00.687ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
60152025-04-30T06:44:00.688ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
60162025-04-30T06:44:00.691ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
60172025-04-30T06:44:00.691ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
60182025-04-30T06:44:00.692ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
60192025-04-30T06:44:00.692ZDEBGtest_utilization (BackgroundTasks): inventory_prune_one: nothing eligible for removal (too few) background_task = inventory_collection candidates = [(13d8b8d9-6c6b-47a6-9190-952544edb75b, 0)]
60202025-04-30T06:44:00.692ZTRACtest_utilization (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal
60212025-04-30T06:44:00.694ZTRACtest_utilization (dns): buffer buffer = Length: 67 (0x43) bytes\n0000: 30 55 01 00 00 01 00 00 00 00 00 01 04 5f 6d 67 0U..........._mg\n0010: 73 04 5f 74 63 70 0d 63 6f 6e 74 72 6f 6c 2d 70 s._tcp.control-p\n0020: 6c 61 6e 65 05 6f 78 69 64 65 08 69 6e 74 65 72 lane.oxide.inter\n0030: 6e 61 6c 00 00 21 00 01 00 00 29 04 d0 00 00 00 nal..!....).....\n0040: 00 00 00 ... kind = dns peer_addr = [::1]:55848 req_id = 492a9aa9-50a4-44a4-89dc-b984f2672f60
60222025-04-30T06:44:00.694ZDEBGtest_utilization (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 12373,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_mgs._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_mgs._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 4,\n 95,\n 109,\n 103,\n 115,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:55848 req_id = 492a9aa9-50a4-44a4-89dc-b984f2672f60
60232025-04-30T06:44:00.694ZDEBGtest_utilization (store): query key key = _mgs._tcp
60242025-04-30T06:44:00.694ZDEBGtest_utilization (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
60252025-04-30T06:44:00.694ZDEBGtest_utilization (dns): dns response additional_records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal"), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] kind = dns peer_addr = [::1]:55848 query = LowerQuery { name: LowerName(Name("_mgs._tcp.control-plane.oxide.internal.")), original: Query { name: Name("_mgs._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN } } records = [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 48759, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal") })) }] req_id = 492a9aa9-50a4-44a4-89dc-b984f2672f60
60262025-04-30T06:44:00.694ZDEBGtest_utilization (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgs._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 48759, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }], valid_until: Instant { tv_sec: 4305, tv_nsec: 493239365 } })
60272025-04-30T06:44:00.694ZTRACtest_utilization (DnsResolver): lookup_service_targets: looking up SRV target
60282025-04-30T06:44:00.694ZTRACtest_utilization (dns): buffer buffer = Length: 108 (0x6c) bytes\n0000: ba a7 01 00 00 01 00 00 00 00 00 01 2d 64 65 6e ............-den\n0010: 64 72 69 74 65 2d 62 36 64 36 35 33 34 31 2d 31 drite-b6d65341-1\n0020: 36 37 63 2d 34 31 64 66 2d 39 62 35 63 2d 34 31 67c-41df-9b5c-41\n0030: 63 64 65 64 39 39 63 32 32 39 04 68 6f 73 74 0d cded99c229.host.\n0040: 63 6f 6e 74 72 6f 6c 2d 70 6c 61 6e 65 05 6f 78 control-plane.ox\n0050: 69 64 65 08 69 6e 74 65 72 6e 61 6c 00 00 1c 00 ide.internal....\n0060: 01 00 00 29 04 d0 00 00 00 00 00 00 ...)........ kind = dns peer_addr = [::1]:49193 req_id = 6f0be8c2-0043-4943-a3e6-cf46e7a7c672
60292025-04-30T06:44:00.695ZDEBGtest_utilization (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 47783,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n query_type: AAAA,\n query_class: IN,\n },\n },\n original: [\n 45,\n 100,\n 101,\n 110,\n 100,\n 114,\n 105,\n 116,\n 101,\n 45,\n 98,\n 54,\n 100,\n 54,\n 53,\n 51,\n 52,\n 49,\n 45,\n 49,\n 54,\n 55,\n 99,\n 45,\n 52,\n 49,\n 100,\n 102,\n 45,\n 57,\n 98,\n 53,\n 99,\n 45,\n 52,\n 49,\n 99,\n 100,\n 101,\n 100,\n 57,\n 57,\n 99,\n 50,\n 50,\n 57,\n 4,\n 104,\n 111,\n 115,\n 116,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 28,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:49193 req_id = 6f0be8c2-0043-4943-a3e6-cf46e7a7c672
60302025-04-30T06:44:00.695ZDEBGtest_utilization (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
60312025-04-30T06:44:00.695ZDEBGtest_utilization (dns): dns response additional_records = [] kind = dns peer_addr = [::1]:49193 query = LowerQuery { name: LowerName(Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.")), original: Query { name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), query_type: AAAA, query_class: IN } } records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] req_id = 6f0be8c2-0043-4943-a3e6-cf46e7a7c672
60322025-04-30T06:44:00.909ZTRACtest_utilization (DnsResolver): lookup_all_socket_v6 srv dns_name = _clickhouse-admin-keeper._tcp.control-plane.oxide.internal
60332025-04-30T06:44:00.912ZTRACtest_utilization (dns): buffer buffer = Length: 87 (0x57) bytes\n0000: 6f 1b 01 00 00 01 00 00 00 00 00 01 18 5f 63 6c o............_cl\n0010: 69 63 6b 68 6f 75 73 65 2d 61 64 6d 69 6e 2d 6b ickhouse-admin-k\n0020: 65 65 70 65 72 04 5f 74 63 70 0d 63 6f 6e 74 72 eeper._tcp.contr\n0030: 6f 6c 2d 70 6c 61 6e 65 05 6f 78 69 64 65 08 69 ol-plane.oxide.i\n0040: 6e 74 65 72 6e 61 6c 00 00 21 00 01 00 00 29 04 nternal..!....).\n0050: d0 00 00 00 00 00 00 ....... kind = dns peer_addr = [::1]:51490 req_id = 84011f01-12eb-4a91-9041-0d1d322b5e2a
60342025-04-30T06:44:00.912ZDEBGtest_utilization (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 28443,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 24,\n 95,\n 99,\n 108,\n 105,\n 99,\n 107,\n 104,\n 111,\n 117,\n 115,\n 101,\n 45,\n 97,\n 100,\n 109,\n 105,\n 110,\n 45,\n 107,\n 101,\n 101,\n 112,\n 101,\n 114,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:51490 req_id = 84011f01-12eb-4a91-9041-0d1d322b5e2a
60352025-04-30T06:44:00.912ZDEBGtest_utilization (store): query key key = _clickhouse-admin-keeper._tcp
60362025-04-30T06:44:00.913ZERROtest_utilization (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 28443, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 1, }, query: WireQuery { query: LowerQuery { name: LowerName( Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), ), original: Query { name: Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN, }, }, original: [ 24, 95, 99, 108, 105, 99, 107, 104, 111, 117, 115, 101, 45, 97, 100, 109, 105, 110, 45, 107, 101, 101, 112, 101, 114, 4, 95, 116, 99, 112, 13, 99, 111, 110, 116, 114, 111, 108, 45, 112, 108, 97, 110, 101, 5, 111, 120, 105, 100, 101, 8, 105, 110, 116, 101, 114, 110, 97, 108, 0, 0, 33, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: Some( Edns { rcode_high: 0, version: 0, dnssec_ok: false, max_payload: 1232, options: OPT { options: {}, }, }, ), } NXDOMAIN: no records found for name: "_clickhouse-admin-keeper._tcp.control-plane.oxide.internal." kind = dns peer_addr = [::1]:51490 req_id = 84011f01-12eb-4a91-9041-0d1d322b5e2a
60372025-04-30T06:44:00.964ZDEBGtest_utilization (BackgroundTasks): begin collection background_task = inventory_collection
60382025-04-30T06:44:00.964ZDEBGtest_utilization (BackgroundTasks): begin collection from MGS background_task = inventory_collection mgs_url = http://[::1]:48759
60392025-04-30T06:44:00.964ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/ignition
60402025-04-30T06:44:00.968ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/ignition", status: 200, headers: {"content-type": "application/json", "x-request-id": "a5dca798-3b27-4009-9e89-04e504b5a365", "content-length": "835", "date": "Wed, 30 Apr 2025 06:44:00 GMT"} })
60412025-04-30T06:44:00.968ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0
60422025-04-30T06:44:00.970ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "34a77321-d084-45e9-96c6-71ffa9ddd7cd", "content-length": "734", "date": "Wed, 30 Apr 2025 06:44:00 GMT"} })
60432025-04-30T06:44:00.971ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0/component/sp/caboose?firmware_slot=0
60442025-04-30T06:44:00.975ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "dbbf37d6-732b-4085-b343-856db61ed2ab", "content-length": "111", "date": "Wed, 30 Apr 2025 06:44:00 GMT"} })
60452025-04-30T06:44:00.975ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0/component/sp/caboose?firmware_slot=1
60462025-04-30T06:44:00.982ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "f0c8d65c-6dd0-47ad-8788-04d5a9f4e7e0", "content-length": "111", "date": "Wed, 30 Apr 2025 06:44:00 GMT"} })
60472025-04-30T06:44:00.982ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0/component/rot/caboose?firmware_slot=0
60482025-04-30T06:44:00.988ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "69b83d0c-3b79-4757-9629-88ceb16db6e7", "content-length": "170", "date": "Wed, 30 Apr 2025 06:44:00 GMT"} })
60492025-04-30T06:44:00.988ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0/component/rot/caboose?firmware_slot=1
60502025-04-30T06:44:00.993ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "d919b0b6-755d-4ee4-ab04-c250673b9dea", "content-length": "170", "date": "Wed, 30 Apr 2025 06:44:00 GMT"} })
60512025-04-30T06:44:00.993ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0/component/stage0/caboose?firmware_slot=0
60522025-04-30T06:44:00.995ZDEBGtest_utilization (collection-task): sent timer-based collection request to the collection task collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
60532025-04-30T06:44:00.996ZDEBGtest_utilization (collection-task): collecting from producer collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
60542025-04-30T06:44:00.999ZTRACtest_utilization (collection-task): sent collection request to producer collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
60552025-04-30T06:44:01.001ZDEBGtest_utilization (collection-task): collected results from producer collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 n_results = 1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
60562025-04-30T06:44:01.001ZTRACtest_utilization (collection-task): forwarded results to main collection loop collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
60572025-04-30T06:44:01.001ZTRACtest_utilization (collection-task): top of inner collection loop, waiting for next request collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
60582025-04-30T06:44:01.012ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "cd3324fc-9ae2-4e2e-aedb-d3ac04ef183c", "content-length": "179", "date": "Wed, 30 Apr 2025 06:44:00 GMT"} })
60592025-04-30T06:44:01.012ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0/component/stage0/caboose?firmware_slot=1
60602025-04-30T06:44:01.018ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "c16b30c7-7bbc-4cb6-b86d-ef9612c389bd", "content-length": "179", "date": "Wed, 30 Apr 2025 06:44:00 GMT"} })
60612025-04-30T06:44:01.019ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0/component/rot/cmpa
60622025-04-30T06:44:01.021ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "65714755-aa59-4e12-b8d2-24106b08ce75", "content-length": "702", "date": "Wed, 30 Apr 2025 06:44:00 GMT"} })
60632025-04-30T06:44:01.021ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0/component/rot/cfpa
60642025-04-30T06:44:01.023ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "d32fbc46-ad9e-4fde-ba76-ef4a3a8e0354", "content-length": "727", "date": "Wed, 30 Apr 2025 06:44:00 GMT"} })
60652025-04-30T06:44:01.024ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0/component/rot/cfpa
60662025-04-30T06:44:01.027ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "f32294dd-457c-4890-9ebf-2d59b2491824", "content-length": "729", "date": "Wed, 30 Apr 2025 06:44:00 GMT"} })
60672025-04-30T06:44:01.027ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0/component/rot/cfpa
60682025-04-30T06:44:01.030ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "0fc68019-e900-4a50-ac9b-b67d84a89146", "content-length": "728", "date": "Wed, 30 Apr 2025 06:44:00 GMT"} })
60692025-04-30T06:44:01.031ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1
60702025-04-30T06:44:01.032ZDEBGtest_utilization: attempt to fetch latest inventory collection result = Ok("found")
60712025-04-30T06:44:01.040ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1", status: 200, headers: {"content-type": "application/json", "x-request-id": "df2307e8-7b88-4639-a944-4c845a3bab77", "content-length": "734", "date": "Wed, 30 Apr 2025 06:44:00 GMT"} })
60722025-04-30T06:44:01.040ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1/component/sp/caboose?firmware_slot=0
60732025-04-30T06:44:01.042ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "3f65c63f-f936-4e8f-8b26-0973713a7292", "content-length": "111", "date": "Wed, 30 Apr 2025 06:44:00 GMT"} })
60742025-04-30T06:44:01.042ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1/component/sp/caboose?firmware_slot=1
60752025-04-30T06:44:01.044ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "ee40e19e-3566-47ce-be22-1a8d91f4ad0e", "content-length": "111", "date": "Wed, 30 Apr 2025 06:44:00 GMT"} })
60762025-04-30T06:44:01.045ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1/component/rot/caboose?firmware_slot=0
60772025-04-30T06:44:01.048ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "ba1fcbee-df0d-4f62-9703-367c65003467", "content-length": "170", "date": "Wed, 30 Apr 2025 06:44:00 GMT"} })
60782025-04-30T06:44:01.048ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1/component/rot/caboose?firmware_slot=1
60792025-04-30T06:44:01.051ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "abd9ebe3-2b13-473a-8ca1-fc123b5bae55", "content-length": "170", "date": "Wed, 30 Apr 2025 06:44:00 GMT"} })
60802025-04-30T06:44:01.052ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1/component/stage0/caboose?firmware_slot=0
60812025-04-30T06:44:01.054ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "2dc8027d-d98b-43a2-aaf0-e20259296e04", "content-length": "179", "date": "Wed, 30 Apr 2025 06:44:01 GMT"} })
60822025-04-30T06:44:01.054ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1/component/stage0/caboose?firmware_slot=1
60832025-04-30T06:44:01.056ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "2d18d429-bc42-4492-b09f-6340debebb0c", "content-length": "179", "date": "Wed, 30 Apr 2025 06:44:01 GMT"} })
60842025-04-30T06:44:01.056ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1/component/rot/cmpa
60852025-04-30T06:44:01.057ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "5757e207-338a-4a61-82da-83161df4b89d", "content-length": "702", "date": "Wed, 30 Apr 2025 06:44:01 GMT"} })
60862025-04-30T06:44:01.057ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1/component/rot/cfpa
60872025-04-30T06:44:01.058ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "5bb5ccff-2578-4910-815c-8df6fc158c4a", "content-length": "727", "date": "Wed, 30 Apr 2025 06:44:01 GMT"} })
60882025-04-30T06:44:01.058ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1/component/rot/cfpa
60892025-04-30T06:44:01.059ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "dcc206ff-ab44-4ded-995a-c49598e8eef1", "content-length": "729", "date": "Wed, 30 Apr 2025 06:44:01 GMT"} })
60902025-04-30T06:44:01.059ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1/component/rot/cfpa
60912025-04-30T06:44:01.060ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "dfebabb9-8e12-44fb-81ee-e5131ddd2bb4", "content-length": "728", "date": "Wed, 30 Apr 2025 06:44:01 GMT"} })
60922025-04-30T06:44:01.061ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0
60932025-04-30T06:44:01.062ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "cd3f647b-5107-41cb-99a8-6b8e05435fe2", "content-length": "723", "date": "Wed, 30 Apr 2025 06:44:01 GMT"} })
60942025-04-30T06:44:01.062ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0/component/sp/caboose?firmware_slot=0
60952025-04-30T06:44:01.065ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "b404aa50-92fb-4fe3-93f7-113aa5ee1873", "content-length": "109", "date": "Wed, 30 Apr 2025 06:44:01 GMT"} })
60962025-04-30T06:44:01.065ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0/component/sp/caboose?firmware_slot=1
60972025-04-30T06:44:01.079ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "a642d2d8-10d3-4f07-bc0f-5671d5d1de73", "content-length": "109", "date": "Wed, 30 Apr 2025 06:44:01 GMT"} })
60982025-04-30T06:44:01.079ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0/component/rot/caboose?firmware_slot=0
60992025-04-30T06:44:01.082ZTRACtest_utilization (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory
61002025-04-30T06:44:01.082ZTRACtest_utilization (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
61012025-04-30T06:44:01.082ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
61022025-04-30T06:44:01.083ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
61032025-04-30T06:44:01.083ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
61042025-04-30T06:44:01.084ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "447d7d75-4aa4-4e95-9683-de1895ee8569", "content-length": "169", "date": "Wed, 30 Apr 2025 06:44:01 GMT"} })
61052025-04-30T06:44:01.084ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0/component/rot/caboose?firmware_slot=1
61062025-04-30T06:44:01.092ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
61072025-04-30T06:44:01.099ZDEBGtest_utilization (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory result = Ok(())
61082025-04-30T06:44:01.099ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
61092025-04-30T06:44:01.099ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
61102025-04-30T06:44:01.102ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
61112025-04-30T06:44:01.105ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "7eb6e89e-2202-40d3-a85a-1b0bc4bb3318", "content-length": "169", "date": "Wed, 30 Apr 2025 06:44:01 GMT"} })
61122025-04-30T06:44:01.106ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0/component/stage0/caboose?firmware_slot=0
61132025-04-30T06:44:01.106ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
61142025-04-30T06:44:01.106ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
61152025-04-30T06:44:01.107ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
61162025-04-30T06:44:01.110ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "95a0a880-cc4e-49a1-9a8b-b9ba5dad0d36", "content-length": "178", "date": "Wed, 30 Apr 2025 06:44:01 GMT"} })
61172025-04-30T06:44:01.110ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0/component/stage0/caboose?firmware_slot=1
61182025-04-30T06:44:01.114ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "3ce018b0-12bc-41dc-b86c-af6222e3ff46", "content-length": "178", "date": "Wed, 30 Apr 2025 06:44:01 GMT"} })
61192025-04-30T06:44:01.114ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0/component/rot/cmpa
61202025-04-30T06:44:01.114ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "4217ebb1-ca97-4ece-9d43-372ea7317e5e", "content-length": "702", "date": "Wed, 30 Apr 2025 06:44:01 GMT"} })
61212025-04-30T06:44:01.114ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0/component/rot/cfpa
61222025-04-30T06:44:01.116ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "c13d2a1b-7205-455d-932f-21042349d1a9", "content-length": "727", "date": "Wed, 30 Apr 2025 06:44:01 GMT"} })
61232025-04-30T06:44:01.116ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0/component/rot/cfpa
61242025-04-30T06:44:01.121ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "2ff0af4c-62c1-40aa-a055-2be179146199", "content-length": "729", "date": "Wed, 30 Apr 2025 06:44:01 GMT"} })
61252025-04-30T06:44:01.121ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0/component/rot/cfpa
61262025-04-30T06:44:01.123ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "7fa42ee9-28da-4fab-819b-7d913b215c02", "content-length": "728", "date": "Wed, 30 Apr 2025 06:44:01 GMT"} })
61272025-04-30T06:44:01.123ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1
61282025-04-30T06:44:01.128ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1", status: 200, headers: {"content-type": "application/json", "x-request-id": "62337da6-aec2-4243-9a9b-dcf2c99324ef", "content-length": "723", "date": "Wed, 30 Apr 2025 06:44:01 GMT"} })
61292025-04-30T06:44:01.128ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1/component/sp/caboose?firmware_slot=0
61302025-04-30T06:44:01.130ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "be9263a4-0431-49dd-90f1-96194dc1ca00", "content-length": "109", "date": "Wed, 30 Apr 2025 06:44:01 GMT"} })
61312025-04-30T06:44:01.130ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1/component/sp/caboose?firmware_slot=1
61322025-04-30T06:44:01.134ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "4537de05-c69e-476b-98b7-d16df5a1dae9", "content-length": "109", "date": "Wed, 30 Apr 2025 06:44:01 GMT"} })
61332025-04-30T06:44:01.134ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1/component/rot/caboose?firmware_slot=0
61342025-04-30T06:44:01.141ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "abd617c6-c5ea-4df6-9ca7-eeeca9a56a6f", "content-length": "169", "date": "Wed, 30 Apr 2025 06:44:01 GMT"} })
61352025-04-30T06:44:01.141ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1/component/rot/caboose?firmware_slot=1
61362025-04-30T06:44:01.145ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "f1b3c800-2529-4cb7-8a90-55d33b611abd", "content-length": "169", "date": "Wed, 30 Apr 2025 06:44:01 GMT"} })
61372025-04-30T06:44:01.146ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1/component/stage0/caboose?firmware_slot=0
61382025-04-30T06:44:01.151ZDEBGtest_utilization: attempt to fetch latest inventory collection result = Ok("found")
61392025-04-30T06:44:01.173ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "1d1e131d-1f1d-44c5-8280-0a37546168c7", "content-length": "178", "date": "Wed, 30 Apr 2025 06:44:01 GMT"} })
61402025-04-30T06:44:01.173ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1/component/stage0/caboose?firmware_slot=1
61412025-04-30T06:44:01.205ZTRACtest_utilization (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory
61422025-04-30T06:44:01.205ZTRACtest_utilization (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
61432025-04-30T06:44:01.205ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
61442025-04-30T06:44:01.205ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
61452025-04-30T06:44:01.206ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
61462025-04-30T06:44:01.209ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "71a061c5-eb43-49c0-b303-c11697ee9324", "content-length": "178", "date": "Wed, 30 Apr 2025 06:44:01 GMT"} })
61472025-04-30T06:44:01.209ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1/component/rot/cmpa
61482025-04-30T06:44:01.210ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
61492025-04-30T06:44:01.218ZDEBGtest_utilization (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory result = Ok(())
61502025-04-30T06:44:01.219ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
61512025-04-30T06:44:01.219ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
61522025-04-30T06:44:01.219ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
61532025-04-30T06:44:01.219ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "ec0dab10-c5e5-44fa-ab34-4cdc9e781464", "content-length": "702", "date": "Wed, 30 Apr 2025 06:44:01 GMT"} })
61542025-04-30T06:44:01.219ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1/component/rot/cfpa
61552025-04-30T06:44:01.225ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
61562025-04-30T06:44:01.225ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
61572025-04-30T06:44:01.225ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
61582025-04-30T06:44:01.226ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "77ee22ba-1675-402d-b8ee-33eb37ce3dc7", "content-length": "727", "date": "Wed, 30 Apr 2025 06:44:01 GMT"} })
61592025-04-30T06:44:01.226ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1/component/rot/cfpa
61602025-04-30T06:44:01.228ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "270f4304-d398-413e-8094-94d967bf1bc2", "content-length": "729", "date": "Wed, 30 Apr 2025 06:44:01 GMT"} })
61612025-04-30T06:44:01.228ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1/component/rot/cfpa
61622025-04-30T06:44:01.230ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "59d01e15-5bd5-4bef-9019-52f26cf1bd98", "content-length": "728", "date": "Wed, 30 Apr 2025 06:44:01 GMT"} })
61632025-04-30T06:44:01.230ZTRACtest_utilization (BackgroundTasks): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet
61642025-04-30T06:44:01.230ZTRACtest_utilization (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
61652025-04-30T06:44:01.230ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
61662025-04-30T06:44:01.230ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
61672025-04-30T06:44:01.231ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
61682025-04-30T06:44:01.265ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
61692025-04-30T06:44:01.268ZDEBGtest_utilization (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet result = Ok(())
61702025-04-30T06:44:01.270ZTRACtest_utilization (BackgroundTasks): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet
61712025-04-30T06:44:01.270ZTRACtest_utilization (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
61722025-04-30T06:44:01.270ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
61732025-04-30T06:44:01.270ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
61742025-04-30T06:44:01.271ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
61752025-04-30T06:44:01.278ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
61762025-04-30T06:44:01.298ZDEBGtest_utilization (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet result = Ok(())
61772025-04-30T06:44:01.305ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
61782025-04-30T06:44:01.306ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
61792025-04-30T06:44:01.307ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
61802025-04-30T06:44:01.398ZDEBGtest_utilization (BackgroundTasks): begin collection from Sled Agent background_task = inventory_collection sled_agent_url = http://[::1]:39365
61812025-04-30T06:44:01.398ZDEBGtest_utilization (BackgroundTasks): client request SledAgent = http://[::1]:39365 background_task = inventory_collection body = None method = GET uri = http://[::1]:39365/inventory
61822025-04-30T06:44:01.399ZINFOtest_utilization (dropshot): accepted connection local_addr = [::1]:39365 remote_addr = [::1]:41511 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
61832025-04-30T06:44:01.399ZDEBGtest_utilization (results-sink): inserting 1 samples into database collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
61842025-04-30T06:44:01.399ZDEBGtest_utilization (results-sink-cluster): inserting 1 samples into database collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
61852025-04-30T06:44:01.401ZDEBGtest_utilization (clickhouse-client): unrolling 1 total samples collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 4d1485f0-110e-4161-9a33-c4610d68be5d
61862025-04-30T06:44:01.403ZTRACtest_utilization (clickhouse-client): inserting data collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 4d1485f0-110e-4161-9a33-c4610d68be5d n_columns = 4 n_rows = 2 sql = INSERT INTO oximeter.fields_string FORMAT Native
61872025-04-30T06:44:01.403ZTRACtest_utilization (dropshot): incoming request local_addr = [::1]:39365 method = GET remote_addr = [::1]:41511 req_id = 463b8166-3332-4c6f-bf1a-f3b9efcd0987 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /inventory
61882025-04-30T06:44:01.405ZINFOtest_utilization (dropshot): request completed latency_us = 1721 local_addr = [::1]:39365 method = GET remote_addr = [::1]:41511 req_id = 463b8166-3332-4c6f-bf1a-f3b9efcd0987 response_code = 200 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /inventory
61892025-04-30T06:44:01.406ZDEBGtest_utilization (BackgroundTasks): client response SledAgent = http://[::1]:39365 background_task = inventory_collection result = Ok(Response { url: "http://[::1]:39365/inventory", status: 200, headers: {"content-type": "application/json", "x-request-id": "463b8166-3332-4c6f-bf1a-f3b9efcd0987", "content-length": "3141", "date": "Wed, 30 Apr 2025 06:44:01 GMT"} })
61902025-04-30T06:44:01.408ZDEBGtest_utilization (BackgroundTasks): begin collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
61912025-04-30T06:44:01.408ZDEBGtest_utilization (BackgroundTasks): end collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
61922025-04-30T06:44:01.408ZDEBGtest_utilization (BackgroundTasks): finished collection background_task = inventory_collection
61932025-04-30T06:44:01.413ZTRACtest_utilization (BackgroundTasks): authorize begin action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory
61942025-04-30T06:44:01.414ZTRACtest_utilization (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
61952025-04-30T06:44:01.414ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
61962025-04-30T06:44:01.414ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
61972025-04-30T06:44:01.415ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
61982025-04-30T06:44:01.416ZDEBGtest_utilization (clickhouse-client): inserted rows into table collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 4d1485f0-110e-4161-9a33-c4610d68be5d n_rows = 2 table_name = fields_string
61992025-04-30T06:44:01.416ZTRACtest_utilization (clickhouse-client): inserting data collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 4d1485f0-110e-4161-9a33-c4610d68be5d n_columns = 4 n_rows = 1 sql = INSERT INTO oximeter.measurements_i64 FORMAT Native
62002025-04-30T06:44:01.437ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
62012025-04-30T06:44:01.441ZDEBGtest_utilization (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory result = Ok(())
62022025-04-30T06:44:01.441ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
62032025-04-30T06:44:01.442ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
62042025-04-30T06:44:01.442ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
62052025-04-30T06:44:01.443ZDEBGtest_utilization (clickhouse-client): inserted rows into table collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 4d1485f0-110e-4161-9a33-c4610d68be5d n_rows = 1 table_name = measurements_i64
62062025-04-30T06:44:01.443ZTRACtest_utilization (results-sink): successfully inserted samples collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
62072025-04-30T06:44:01.449ZDEBGtest_utilization: attempt to fetch latest inventory collection result = Ok("found")
62082025-04-30T06:44:01.500ZTRACtest_utilization (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory
62092025-04-30T06:44:01.500ZTRACtest_utilization (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
62102025-04-30T06:44:01.500ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
62112025-04-30T06:44:01.500ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
62122025-04-30T06:44:01.500ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
62132025-04-30T06:44:01.501ZWARNtest_utilization (results-sink-cluster): failed to insert some results into metric DB collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 error = Failed to check out connection to database: No backends found for this service file = oximeter/collector/src/results_sink.rs:92
62142025-04-30T06:44:01.505ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
62152025-04-30T06:44:01.510ZDEBGtest_utilization (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory result = Ok(())
62162025-04-30T06:44:01.511ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
62172025-04-30T06:44:01.511ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
62182025-04-30T06:44:01.511ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
62192025-04-30T06:44:01.610ZWARNtest_utilization (datastore): Non-retryable transaction failure err = DatabaseError(UniqueViolation, "duplicate key value violates unique constraint \\"inv_caboose_pkey\\"") transaction = inventory_insert_collection
62202025-04-30T06:44:01.611ZWARNtest_utilization (BackgroundTasks): inventory collection failed background_task = inventory_collection error = failed to collect inventory: saving inventory to database: Internal Error: unexpected database error: duplicate key value violates unique constraint "inv_caboose_pkey"
62212025-04-30T06:44:01.611ZDEBGtest_utilization (BackgroundTasks): activation complete background_task = inventory_collection elapsed = 936.122352ms iteration = 4 status = {"error":"failed to collect inventory: saving inventory to database: Internal Error: unexpected database error: duplicate key value violates unique constraint \\"inv_caboose_pkey\\""}
62222025-04-30T06:44:01.611ZDEBGtest_utilization (BackgroundTasks): activating background_task = inventory_collection iteration = 5 reason = Signaled
62232025-04-30T06:44:01.611ZTRACtest_utilization (BackgroundTasks): authorize begin action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory
62242025-04-30T06:44:01.611ZTRACtest_utilization (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
62252025-04-30T06:44:01.611ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
62262025-04-30T06:44:01.611ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
62272025-04-30T06:44:01.611ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
62282025-04-30T06:44:01.615ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
62292025-04-30T06:44:01.619ZDEBGtest_utilization (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory result = Ok(())
62302025-04-30T06:44:01.619ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
62312025-04-30T06:44:01.619ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
62322025-04-30T06:44:01.619ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
62332025-04-30T06:44:01.620ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
62342025-04-30T06:44:01.620ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
62352025-04-30T06:44:01.620ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
62362025-04-30T06:44:01.623ZDEBGtest_utilization (BackgroundTasks): inventory_prune_one: nothing eligible for removal (too few) background_task = inventory_collection candidates = [(13d8b8d9-6c6b-47a6-9190-952544edb75b, 0)]
62372025-04-30T06:44:01.624ZTRACtest_utilization (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal
62382025-04-30T06:44:01.626ZTRACtest_utilization (dns): buffer buffer = Length: 67 (0x43) bytes\n0000: ba 62 01 00 00 01 00 00 00 00 00 01 04 5f 6d 67 .b..........._mg\n0010: 73 04 5f 74 63 70 0d 63 6f 6e 74 72 6f 6c 2d 70 s._tcp.control-p\n0020: 6c 61 6e 65 05 6f 78 69 64 65 08 69 6e 74 65 72 lane.oxide.inter\n0030: 6e 61 6c 00 00 21 00 01 00 00 29 04 d0 00 00 00 nal..!....).....\n0040: 00 00 00 ... kind = dns peer_addr = [::1]:52835 req_id = 9645859c-ff45-4d89-a80e-9ec771d9aa9d
62392025-04-30T06:44:01.627ZDEBGtest_utilization (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 47714,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_mgs._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_mgs._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 4,\n 95,\n 109,\n 103,\n 115,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:52835 req_id = 9645859c-ff45-4d89-a80e-9ec771d9aa9d
62402025-04-30T06:44:01.627ZDEBGtest_utilization (store): query key key = _mgs._tcp
62412025-04-30T06:44:01.627ZDEBGtest_utilization (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
62422025-04-30T06:44:01.627ZDEBGtest_utilization (dns): dns response additional_records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal"), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] kind = dns peer_addr = [::1]:52835 query = LowerQuery { name: LowerName(Name("_mgs._tcp.control-plane.oxide.internal.")), original: Query { name: Name("_mgs._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN } } records = [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 48759, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal") })) }] req_id = 9645859c-ff45-4d89-a80e-9ec771d9aa9d
62432025-04-30T06:44:01.628ZDEBGtest_utilization (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgs._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 48759, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }], valid_until: Instant { tv_sec: 4306, tv_nsec: 427187926 } })
62442025-04-30T06:44:01.628ZTRACtest_utilization (DnsResolver): lookup_service_targets: looking up SRV target
62452025-04-30T06:44:01.629ZTRACtest_utilization (dns): buffer buffer = Length: 108 (0x6c) bytes\n0000: 08 09 01 00 00 01 00 00 00 00 00 01 2d 64 65 6e ............-den\n0010: 64 72 69 74 65 2d 62 36 64 36 35 33 34 31 2d 31 drite-b6d65341-1\n0020: 36 37 63 2d 34 31 64 66 2d 39 62 35 63 2d 34 31 67c-41df-9b5c-41\n0030: 63 64 65 64 39 39 63 32 32 39 04 68 6f 73 74 0d cded99c229.host.\n0040: 63 6f 6e 74 72 6f 6c 2d 70 6c 61 6e 65 05 6f 78 control-plane.ox\n0050: 69 64 65 08 69 6e 74 65 72 6e 61 6c 00 00 1c 00 ide.internal....\n0060: 01 00 00 29 04 d0 00 00 00 00 00 00 ...)........ kind = dns peer_addr = [::1]:50452 req_id = d5defc1d-bbad-4ded-a35c-70b436fd7fab
62462025-04-30T06:44:01.629ZDEBGtest_utilization (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 2057,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n query_type: AAAA,\n query_class: IN,\n },\n },\n original: [\n 45,\n 100,\n 101,\n 110,\n 100,\n 114,\n 105,\n 116,\n 101,\n 45,\n 98,\n 54,\n 100,\n 54,\n 53,\n 51,\n 52,\n 49,\n 45,\n 49,\n 54,\n 55,\n 99,\n 45,\n 52,\n 49,\n 100,\n 102,\n 45,\n 57,\n 98,\n 53,\n 99,\n 45,\n 52,\n 49,\n 99,\n 100,\n 101,\n 100,\n 57,\n 57,\n 99,\n 50,\n 50,\n 57,\n 4,\n 104,\n 111,\n 115,\n 116,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 28,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:50452 req_id = d5defc1d-bbad-4ded-a35c-70b436fd7fab
62472025-04-30T06:44:01.629ZDEBGtest_utilization (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
62482025-04-30T06:44:01.629ZDEBGtest_utilization (dns): dns response additional_records = [] kind = dns peer_addr = [::1]:50452 query = LowerQuery { name: LowerName(Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.")), original: Query { name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), query_type: AAAA, query_class: IN } } records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] req_id = d5defc1d-bbad-4ded-a35c-70b436fd7fab
62492025-04-30T06:44:01.680ZTRACtest_utilization (DnsResolver): lookup_all_socket_v6 srv dns_name = _clickhouse-admin-keeper._tcp.control-plane.oxide.internal
62502025-04-30T06:44:01.696ZTRACtest_utilization (dns): buffer buffer = Length: 87 (0x57) bytes\n0000: 93 17 01 00 00 01 00 00 00 00 00 01 18 5f 63 6c ............._cl\n0010: 69 63 6b 68 6f 75 73 65 2d 61 64 6d 69 6e 2d 6b ickhouse-admin-k\n0020: 65 65 70 65 72 04 5f 74 63 70 0d 63 6f 6e 74 72 eeper._tcp.contr\n0030: 6f 6c 2d 70 6c 61 6e 65 05 6f 78 69 64 65 08 69 ol-plane.oxide.i\n0040: 6e 74 65 72 6e 61 6c 00 00 21 00 01 00 00 29 04 nternal..!....).\n0050: d0 00 00 00 00 00 00 ....... kind = dns peer_addr = [::1]:63417 req_id = 55fc39d4-285e-4f21-b1bf-c702d69456be
62512025-04-30T06:44:01.696ZDEBGtest_utilization (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 37655,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 24,\n 95,\n 99,\n 108,\n 105,\n 99,\n 107,\n 104,\n 111,\n 117,\n 115,\n 101,\n 45,\n 97,\n 100,\n 109,\n 105,\n 110,\n 45,\n 107,\n 101,\n 101,\n 112,\n 101,\n 114,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:63417 req_id = 55fc39d4-285e-4f21-b1bf-c702d69456be
62522025-04-30T06:44:01.697ZDEBGtest_utilization (store): query key key = _clickhouse-admin-keeper._tcp
62532025-04-30T06:44:01.697ZERROtest_utilization (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 37655, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 1, }, query: WireQuery { query: LowerQuery { name: LowerName( Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), ), original: Query { name: Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN, }, }, original: [ 24, 95, 99, 108, 105, 99, 107, 104, 111, 117, 115, 101, 45, 97, 100, 109, 105, 110, 45, 107, 101, 101, 112, 101, 114, 4, 95, 116, 99, 112, 13, 99, 111, 110, 116, 114, 111, 108, 45, 112, 108, 97, 110, 101, 5, 111, 120, 105, 100, 101, 8, 105, 110, 116, 101, 114, 110, 97, 108, 0, 0, 33, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: Some( Edns { rcode_high: 0, version: 0, dnssec_ok: false, max_payload: 1232, options: OPT { options: {}, }, }, ), } NXDOMAIN: no records found for name: "_clickhouse-admin-keeper._tcp.control-plane.oxide.internal." kind = dns peer_addr = [::1]:63417 req_id = 55fc39d4-285e-4f21-b1bf-c702d69456be
62542025-04-30T06:44:01.697ZDEBGtest_utilization (BackgroundTasks): begin collection background_task = inventory_collection
62552025-04-30T06:44:01.697ZDEBGtest_utilization (BackgroundTasks): begin collection from MGS background_task = inventory_collection mgs_url = http://[::1]:48759
62562025-04-30T06:44:01.697ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/ignition
62572025-04-30T06:44:01.701ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/ignition", status: 200, headers: {"content-type": "application/json", "x-request-id": "38c117f9-a09d-47e5-a997-8dd04ced39e7", "content-length": "835", "date": "Wed, 30 Apr 2025 06:44:01 GMT"} })
62582025-04-30T06:44:01.702ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0
62592025-04-30T06:44:01.706ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "8ee21e10-0abe-4dad-a50e-60f90b89631b", "content-length": "734", "date": "Wed, 30 Apr 2025 06:44:01 GMT"} })
62602025-04-30T06:44:01.706ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0/component/sp/caboose?firmware_slot=0
62612025-04-30T06:44:01.712ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "3f2f9e8c-4039-485f-8dd6-76b452db07be", "content-length": "111", "date": "Wed, 30 Apr 2025 06:44:01 GMT"} })
62622025-04-30T06:44:01.719ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0/component/sp/caboose?firmware_slot=1
62632025-04-30T06:44:01.726ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "14b693ba-0aa1-47da-8351-39fff1c40b62", "content-length": "111", "date": "Wed, 30 Apr 2025 06:44:01 GMT"} })
62642025-04-30T06:44:01.726ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0/component/rot/caboose?firmware_slot=0
62652025-04-30T06:44:01.730ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "9c7c7e5a-0eee-4add-9c03-f3b1100e159e", "content-length": "170", "date": "Wed, 30 Apr 2025 06:44:01 GMT"} })
62662025-04-30T06:44:01.730ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0/component/rot/caboose?firmware_slot=1
62672025-04-30T06:44:01.733ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "5fe8af8a-26fd-4a83-94f8-6560355bd1d8", "content-length": "170", "date": "Wed, 30 Apr 2025 06:44:01 GMT"} })
62682025-04-30T06:44:01.733ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0/component/stage0/caboose?firmware_slot=0
62692025-04-30T06:44:01.745ZDEBGtest_utilization: attempt to fetch latest inventory collection result = Ok("found")
62702025-04-30T06:44:01.749ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "aa539c14-7c7f-41a0-bd7b-9ddc3ad00dfe", "content-length": "179", "date": "Wed, 30 Apr 2025 06:44:01 GMT"} })
62712025-04-30T06:44:01.749ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0/component/stage0/caboose?firmware_slot=1
62722025-04-30T06:44:01.751ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "4cb84f17-06a9-465a-b157-5bbc2d55a922", "content-length": "179", "date": "Wed, 30 Apr 2025 06:44:01 GMT"} })
62732025-04-30T06:44:01.751ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0/component/rot/cmpa
62742025-04-30T06:44:01.752ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "6f6bd0bd-29b6-4b40-9efd-150b2129b337", "content-length": "702", "date": "Wed, 30 Apr 2025 06:44:01 GMT"} })
62752025-04-30T06:44:01.752ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0/component/rot/cfpa
62762025-04-30T06:44:01.754ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "578c289c-bca0-4552-ab06-3db02d0ad58e", "content-length": "727", "date": "Wed, 30 Apr 2025 06:44:01 GMT"} })
62772025-04-30T06:44:01.754ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0/component/rot/cfpa
62782025-04-30T06:44:01.755ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "ef0e0ab4-1850-4d6e-b717-0233e893d717", "content-length": "729", "date": "Wed, 30 Apr 2025 06:44:01 GMT"} })
62792025-04-30T06:44:01.755ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0/component/rot/cfpa
62802025-04-30T06:44:01.761ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "f12fb06d-2707-4dff-935d-42955907fdfe", "content-length": "728", "date": "Wed, 30 Apr 2025 06:44:01 GMT"} })
62812025-04-30T06:44:01.762ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1
62822025-04-30T06:44:01.763ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1", status: 200, headers: {"content-type": "application/json", "x-request-id": "cba81084-9295-4599-a04d-99b107f42bbb", "content-length": "734", "date": "Wed, 30 Apr 2025 06:44:01 GMT"} })
62832025-04-30T06:44:01.765ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1/component/sp/caboose?firmware_slot=0
62842025-04-30T06:44:01.768ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "3b569548-e3e9-4b61-a6b2-22f7e91d324d", "content-length": "111", "date": "Wed, 30 Apr 2025 06:44:01 GMT"} })
62852025-04-30T06:44:01.769ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1/component/sp/caboose?firmware_slot=1
62862025-04-30T06:44:01.770ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "2d2e29ba-4fb2-47c1-a945-b6d5d9ab4541", "content-length": "111", "date": "Wed, 30 Apr 2025 06:44:01 GMT"} })
62872025-04-30T06:44:01.771ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1/component/rot/caboose?firmware_slot=0
62882025-04-30T06:44:01.773ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "b71b5fb5-9fff-43f0-8b5a-5865f9c73f2d", "content-length": "170", "date": "Wed, 30 Apr 2025 06:44:01 GMT"} })
62892025-04-30T06:44:01.773ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1/component/rot/caboose?firmware_slot=1
62902025-04-30T06:44:01.777ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "7eefdb2c-5e0b-4f4e-be8c-a5d831dc13df", "content-length": "170", "date": "Wed, 30 Apr 2025 06:44:01 GMT"} })
62912025-04-30T06:44:01.777ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1/component/stage0/caboose?firmware_slot=0
62922025-04-30T06:44:01.783ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "2e3fa2d7-b0cc-4434-83a0-9fd5cc217b2e", "content-length": "179", "date": "Wed, 30 Apr 2025 06:44:01 GMT"} })
62932025-04-30T06:44:01.783ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1/component/stage0/caboose?firmware_slot=1
62942025-04-30T06:44:01.785ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "40e08c06-c1e1-4073-88f9-d3d6c0f825be", "content-length": "179", "date": "Wed, 30 Apr 2025 06:44:01 GMT"} })
62952025-04-30T06:44:01.785ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1/component/rot/cmpa
62962025-04-30T06:44:01.786ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "940b7a92-2c95-4601-9573-84c5bdd06948", "content-length": "702", "date": "Wed, 30 Apr 2025 06:44:01 GMT"} })
62972025-04-30T06:44:01.786ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1/component/rot/cfpa
62982025-04-30T06:44:01.787ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "7c06dd53-c264-4e4b-84c9-3b93ef3e79af", "content-length": "727", "date": "Wed, 30 Apr 2025 06:44:01 GMT"} })
62992025-04-30T06:44:01.787ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1/component/rot/cfpa
63002025-04-30T06:44:01.788ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "257a5cda-d4ea-44b2-b829-b19833fbf103", "content-length": "729", "date": "Wed, 30 Apr 2025 06:44:01 GMT"} })
63012025-04-30T06:44:01.788ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1/component/rot/cfpa
63022025-04-30T06:44:01.790ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "094ab5b7-00c5-40d0-ac47-f98dcc1226b9", "content-length": "728", "date": "Wed, 30 Apr 2025 06:44:01 GMT"} })
63032025-04-30T06:44:01.790ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0
63042025-04-30T06:44:01.791ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "5973d651-a561-4d47-b9b3-693d2f299d5a", "content-length": "723", "date": "Wed, 30 Apr 2025 06:44:01 GMT"} })
63052025-04-30T06:44:01.791ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0/component/sp/caboose?firmware_slot=0
63062025-04-30T06:44:01.793ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "9e8dc29a-3443-46f4-9007-758fb3a04885", "content-length": "109", "date": "Wed, 30 Apr 2025 06:44:01 GMT"} })
63072025-04-30T06:44:01.793ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0/component/sp/caboose?firmware_slot=1
63082025-04-30T06:44:01.798ZTRACtest_utilization (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory
63092025-04-30T06:44:01.798ZTRACtest_utilization (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
63102025-04-30T06:44:01.798ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
63112025-04-30T06:44:01.798ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
63122025-04-30T06:44:01.799ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
63132025-04-30T06:44:01.800ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "149faa67-d53e-43a6-9de5-0678c2f0eaa6", "content-length": "109", "date": "Wed, 30 Apr 2025 06:44:01 GMT"} })
63142025-04-30T06:44:01.800ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0/component/rot/caboose?firmware_slot=0
63152025-04-30T06:44:01.802ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
63162025-04-30T06:44:01.807ZDEBGtest_utilization (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory result = Ok(())
63172025-04-30T06:44:01.807ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
63182025-04-30T06:44:01.807ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
63192025-04-30T06:44:01.808ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
63202025-04-30T06:44:01.810ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "b0dc1975-3c26-4ff3-a3f4-f0997b104189", "content-length": "169", "date": "Wed, 30 Apr 2025 06:44:01 GMT"} })
63212025-04-30T06:44:01.810ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0/component/rot/caboose?firmware_slot=1
63222025-04-30T06:44:01.812ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
63232025-04-30T06:44:01.812ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
63242025-04-30T06:44:01.813ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
63252025-04-30T06:44:01.823ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "ae77a3fa-850b-4487-afc5-ec890a59dcc0", "content-length": "169", "date": "Wed, 30 Apr 2025 06:44:01 GMT"} })
63262025-04-30T06:44:01.823ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0/component/stage0/caboose?firmware_slot=0
63272025-04-30T06:44:01.827ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "d532bbb9-ce53-4868-b162-680f6d17f7f2", "content-length": "178", "date": "Wed, 30 Apr 2025 06:44:01 GMT"} })
63282025-04-30T06:44:01.827ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0/component/stage0/caboose?firmware_slot=1
63292025-04-30T06:44:01.830ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "6d3070f3-3aeb-401a-a2cb-fe4824f89d9c", "content-length": "178", "date": "Wed, 30 Apr 2025 06:44:01 GMT"} })
63302025-04-30T06:44:01.830ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0/component/rot/cmpa
63312025-04-30T06:44:01.831ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "f2c05364-c07e-4511-9f70-4fe917551d6f", "content-length": "702", "date": "Wed, 30 Apr 2025 06:44:01 GMT"} })
63322025-04-30T06:44:01.831ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0/component/rot/cfpa
63332025-04-30T06:44:01.832ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "e1532693-3a8a-44af-9f77-0732b9bf75ec", "content-length": "727", "date": "Wed, 30 Apr 2025 06:44:01 GMT"} })
63342025-04-30T06:44:01.832ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0/component/rot/cfpa
63352025-04-30T06:44:01.833ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "e4f8cfcc-4b32-4332-9d36-8be4bbe6bc24", "content-length": "729", "date": "Wed, 30 Apr 2025 06:44:01 GMT"} })
63362025-04-30T06:44:01.834ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0/component/rot/cfpa
63372025-04-30T06:44:01.835ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "fe789633-cd26-4784-b247-11588ca98dc2", "content-length": "728", "date": "Wed, 30 Apr 2025 06:44:01 GMT"} })
63382025-04-30T06:44:01.835ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1
63392025-04-30T06:44:01.836ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1", status: 200, headers: {"content-type": "application/json", "x-request-id": "e45d7e55-6a06-4e06-84c9-8be96876ed84", "content-length": "723", "date": "Wed, 30 Apr 2025 06:44:01 GMT"} })
63402025-04-30T06:44:01.836ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1/component/sp/caboose?firmware_slot=0
63412025-04-30T06:44:01.840ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "ecc885fe-6d7b-4450-a7a4-2daf1f9ed964", "content-length": "109", "date": "Wed, 30 Apr 2025 06:44:01 GMT"} })
63422025-04-30T06:44:01.840ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1/component/sp/caboose?firmware_slot=1
63432025-04-30T06:44:01.844ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "cd49beaa-4ba2-4b41-9817-c6787611b2ae", "content-length": "109", "date": "Wed, 30 Apr 2025 06:44:01 GMT"} })
63442025-04-30T06:44:01.845ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1/component/rot/caboose?firmware_slot=0
63452025-04-30T06:44:01.851ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "05a9ce57-b0bb-422e-93d8-8ff1eb14e34a", "content-length": "169", "date": "Wed, 30 Apr 2025 06:44:01 GMT"} })
63462025-04-30T06:44:01.851ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1/component/rot/caboose?firmware_slot=1
63472025-04-30T06:44:01.855ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "6fcb9dd3-a2d5-4973-b69b-8ad693726dd1", "content-length": "169", "date": "Wed, 30 Apr 2025 06:44:01 GMT"} })
63482025-04-30T06:44:01.855ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1/component/stage0/caboose?firmware_slot=0
63492025-04-30T06:44:01.856ZDEBGtest_utilization: attempt to fetch latest inventory collection result = Ok("found")
63502025-04-30T06:44:01.863ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "32c88bb8-82a1-4284-a03f-0756c67c7611", "content-length": "178", "date": "Wed, 30 Apr 2025 06:44:01 GMT"} })
63512025-04-30T06:44:01.865ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1/component/stage0/caboose?firmware_slot=1
63522025-04-30T06:44:01.880ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "89101023-2f6e-44c7-92a3-adbf9b707729", "content-length": "178", "date": "Wed, 30 Apr 2025 06:44:01 GMT"} })
63532025-04-30T06:44:01.892ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1/component/rot/cmpa
63542025-04-30T06:44:01.894ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "a82a2b50-29c9-47da-93c4-8e80a11df345", "content-length": "702", "date": "Wed, 30 Apr 2025 06:44:01 GMT"} })
63552025-04-30T06:44:01.894ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1/component/rot/cfpa
63562025-04-30T06:44:01.895ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "ca9bc03f-40ae-4d6d-b673-d8d0df26d8aa", "content-length": "727", "date": "Wed, 30 Apr 2025 06:44:01 GMT"} })
63572025-04-30T06:44:01.896ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1/component/rot/cfpa
63582025-04-30T06:44:01.897ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "832db7bd-fd86-4ab0-84bf-6db71505306f", "content-length": "729", "date": "Wed, 30 Apr 2025 06:44:01 GMT"} })
63592025-04-30T06:44:01.897ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1/component/rot/cfpa
63602025-04-30T06:44:01.898ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "9c08af3f-5e66-4ecb-8b67-d103b193a4cd", "content-length": "728", "date": "Wed, 30 Apr 2025 06:44:01 GMT"} })
63612025-04-30T06:44:01.906ZTRACtest_utilization (BackgroundTasks): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet
63622025-04-30T06:44:01.906ZTRACtest_utilization (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
63632025-04-30T06:44:01.906ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
63642025-04-30T06:44:01.906ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
63652025-04-30T06:44:01.907ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
63662025-04-30T06:44:01.907ZTRACtest_utilization (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory
63672025-04-30T06:44:01.907ZTRACtest_utilization (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
63682025-04-30T06:44:01.907ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
63692025-04-30T06:44:01.907ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
63702025-04-30T06:44:01.908ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
63712025-04-30T06:44:01.911ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
63722025-04-30T06:44:01.916ZDEBGtest_utilization (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet result = Ok(())
63732025-04-30T06:44:01.916ZTRACtest_utilization (BackgroundTasks): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet
63742025-04-30T06:44:01.916ZTRACtest_utilization (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
63752025-04-30T06:44:01.916ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
63762025-04-30T06:44:01.916ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
63772025-04-30T06:44:01.916ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
63782025-04-30T06:44:01.924ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
63792025-04-30T06:44:01.931ZDEBGtest_utilization (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet result = Ok(())
63802025-04-30T06:44:01.931ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
63812025-04-30T06:44:01.931ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
63822025-04-30T06:44:01.933ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
63832025-04-30T06:44:01.934ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
63842025-04-30T06:44:01.944ZDEBGtest_utilization (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory result = Ok(())
63852025-04-30T06:44:01.944ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
63862025-04-30T06:44:01.944ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
63872025-04-30T06:44:01.945ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
63882025-04-30T06:44:02.030ZDEBGtest_utilization (BackgroundTasks): begin collection from Sled Agent background_task = inventory_collection sled_agent_url = http://[::1]:39365
63892025-04-30T06:44:02.030ZDEBGtest_utilization (BackgroundTasks): client request SledAgent = http://[::1]:39365 background_task = inventory_collection body = None method = GET uri = http://[::1]:39365/inventory
63902025-04-30T06:44:02.031ZINFOtest_utilization (dropshot): accepted connection local_addr = [::1]:39365 remote_addr = [::1]:38412 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
63912025-04-30T06:44:02.031ZINFOtest_utilization (oximeter-agent): refreshing list of producers from Nexus collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 file = oximeter/collector/src/agent.rs:562
63922025-04-30T06:44:02.031ZDEBGtest_utilization (collection-task): sent timer-based collection request to the collection task collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
63932025-04-30T06:44:02.031ZDEBGtest_utilization (collection-task): collecting from producer collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
63942025-04-30T06:44:02.032ZDEBGtest_utilization (oximeter): client request body = None method = GET uri = http://127.0.0.1:32956/metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?sort_by=id_ascending
63952025-04-30T06:44:02.032ZTRACtest_utilization (dropshot_internal): incoming request local_addr = 127.0.0.1:32956 method = GET remote_addr = 127.0.0.1:55797 req_id = c1d7c0db-698f-4dd5-88ea-2eb97c535599 uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?sort_by=id_ascending
63962025-04-30T06:44:02.033ZTRACtest_utilization (dropshot): incoming request local_addr = [::1]:39365 method = GET remote_addr = [::1]:38412 req_id = 5d02ab97-a040-47f2-8e33-a58ce149d4b2 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /inventory
63972025-04-30T06:44:02.033ZTRACtest_utilization (dropshot_internal): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = 127.0.0.1:32956 method = GET remote_addr = 127.0.0.1:55797 req_id = c1d7c0db-698f-4dd5-88ea-2eb97c535599 resource = Database uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?sort_by=id_ascending
63982025-04-30T06:44:02.033ZDEBGtest_utilization (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = 127.0.0.1:32956 method = GET remote_addr = 127.0.0.1:55797 req_id = c1d7c0db-698f-4dd5-88ea-2eb97c535599 roles = RoleSet { roles: {} } uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?sort_by=id_ascending
63992025-04-30T06:44:02.034ZDEBGtest_utilization (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = 127.0.0.1:32956 method = GET remote_addr = 127.0.0.1:55797 req_id = c1d7c0db-698f-4dd5-88ea-2eb97c535599 resource = Database result = Ok(()) uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?sort_by=id_ascending
64002025-04-30T06:44:02.036ZINFOtest_utilization (dropshot): request completed latency_us = 3359 local_addr = [::1]:39365 method = GET remote_addr = [::1]:38412 req_id = 5d02ab97-a040-47f2-8e33-a58ce149d4b2 response_code = 200 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /inventory
64012025-04-30T06:44:02.036ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
64022025-04-30T06:44:02.036ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
64032025-04-30T06:44:02.037ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
64042025-04-30T06:44:02.038ZTRACtest_utilization (collection-task): sent collection request to producer collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
64052025-04-30T06:44:02.039ZDEBGtest_utilization (collection-task): collected results from producer collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 n_results = 1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
64062025-04-30T06:44:02.039ZTRACtest_utilization (collection-task): forwarded results to main collection loop collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
64072025-04-30T06:44:02.039ZTRACtest_utilization (collection-task): top of inner collection loop, waiting for next request collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
64082025-04-30T06:44:02.039ZDEBGtest_utilization (BackgroundTasks): client response SledAgent = http://[::1]:39365 background_task = inventory_collection result = Ok(Response { url: "http://[::1]:39365/inventory", status: 200, headers: {"content-type": "application/json", "x-request-id": "5d02ab97-a040-47f2-8e33-a58ce149d4b2", "content-length": "3141", "date": "Wed, 30 Apr 2025 06:44:01 GMT"} })
64092025-04-30T06:44:02.042ZDEBGtest_utilization (BackgroundTasks): begin collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
64102025-04-30T06:44:02.042ZDEBGtest_utilization (BackgroundTasks): end collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
64112025-04-30T06:44:02.042ZDEBGtest_utilization (BackgroundTasks): finished collection background_task = inventory_collection
64122025-04-30T06:44:02.044ZTRACtest_utilization (BackgroundTasks): authorize begin action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory
64132025-04-30T06:44:02.044ZTRACtest_utilization (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
64142025-04-30T06:44:02.044ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
64152025-04-30T06:44:02.044ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
64162025-04-30T06:44:02.044ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
64172025-04-30T06:44:02.047ZINFOtest_utilization (dropshot_internal): request completed latency_us = 14838 local_addr = 127.0.0.1:32956 method = GET remote_addr = 127.0.0.1:55797 req_id = c1d7c0db-698f-4dd5-88ea-2eb97c535599 response_code = 200 uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?sort_by=id_ascending
64182025-04-30T06:44:02.048ZDEBGtest_utilization (oximeter): client response result = Ok(Response { url: "http://127.0.0.1:32956/metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?sort_by=id_ascending", status: 200, headers: {"content-type": "application/json", "x-request-id": "c1d7c0db-698f-4dd5-88ea-2eb97c535599", "content-length": "405", "date": "Wed, 30 Apr 2025 06:44:01 GMT"} })
64192025-04-30T06:44:02.048ZDEBGtest_utilization (oximeter): client request body = None method = GET uri = http://127.0.0.1:32956/metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJhYzU1ZDhkOS1hMjk4LTQ1MzQtODk4Yi1iZWVlMWM5ZTRlNTIifX0%3D
64202025-04-30T06:44:02.049ZTRACtest_utilization (dropshot_internal): incoming request local_addr = 127.0.0.1:32956 method = GET remote_addr = 127.0.0.1:55797 req_id = d25d9391-a3f9-4760-81d3-6fcbde072b00 uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJhYzU1ZDhkOS1hMjk4LTQ1MzQtODk4Yi1iZWVlMWM5ZTRlNTIifX0%3D
64212025-04-30T06:44:02.049ZTRACtest_utilization (dropshot_internal): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = 127.0.0.1:32956 method = GET remote_addr = 127.0.0.1:55797 req_id = d25d9391-a3f9-4760-81d3-6fcbde072b00 resource = Database uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJhYzU1ZDhkOS1hMjk4LTQ1MzQtODk4Yi1iZWVlMWM5ZTRlNTIifX0%3D
64222025-04-30T06:44:02.049ZDEBGtest_utilization (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = 127.0.0.1:32956 method = GET remote_addr = 127.0.0.1:55797 req_id = d25d9391-a3f9-4760-81d3-6fcbde072b00 roles = RoleSet { roles: {} } uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJhYzU1ZDhkOS1hMjk4LTQ1MzQtODk4Yi1iZWVlMWM5ZTRlNTIifX0%3D
64232025-04-30T06:44:02.050ZDEBGtest_utilization (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = 127.0.0.1:32956 method = GET remote_addr = 127.0.0.1:55797 req_id = d25d9391-a3f9-4760-81d3-6fcbde072b00 resource = Database result = Ok(()) uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJhYzU1ZDhkOS1hMjk4LTQ1MzQtODk4Yi1iZWVlMWM5ZTRlNTIifX0%3D
64242025-04-30T06:44:02.050ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
64252025-04-30T06:44:02.056ZDEBGtest_utilization (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory result = Ok(())
64262025-04-30T06:44:02.057ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
64272025-04-30T06:44:02.057ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
64282025-04-30T06:44:02.058ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
64292025-04-30T06:44:02.061ZINFOtest_utilization (dropshot_internal): request completed latency_us = 11931 local_addr = 127.0.0.1:32956 method = GET remote_addr = 127.0.0.1:55797 req_id = d25d9391-a3f9-4760-81d3-6fcbde072b00 response_code = 200 uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJhYzU1ZDhkOS1hMjk4LTQ1MzQtODk4Yi1iZWVlMWM5ZTRlNTIifX0%3D
64302025-04-30T06:44:02.063ZDEBGtest_utilization (oximeter): client response result = Ok(Response { url: "http://127.0.0.1:32956/metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJhYzU1ZDhkOS1hMjk4LTQ1MzQtODk4Yi1iZWVlMWM5ZTRlNTIifX0%3D", status: 200, headers: {"content-type": "application/json", "x-request-id": "d25d9391-a3f9-4760-81d3-6fcbde072b00", "content-length": "29", "date": "Wed, 30 Apr 2025 06:44:02 GMT"} })
64312025-04-30T06:44:02.063ZTRACtest_utilization (oximeter-agent): ignoring request to update existing metric producer, since the endpoint information is the same as the existing collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
64322025-04-30T06:44:02.064ZTRACtest_utilization (oximeter-agent): ignoring request to update existing metric producer, since the endpoint information is the same as the existing collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = ac55d8d9-a298-4534-898b-beee1c9e4e52
64332025-04-30T06:44:02.064ZINFOtest_utilization (oximeter-agent): refreshed list of producers from Nexus collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 file = oximeter/collector/src/agent.rs:637 n_current_tasks = 2 n_pruned_tasks = 0
64342025-04-30T06:44:02.120ZWARNtest_utilization (datastore): Non-retryable transaction failure err = DatabaseError(UniqueViolation, "duplicate key value violates unique constraint \\"inv_caboose_pkey\\"") transaction = inventory_insert_collection
64352025-04-30T06:44:02.121ZWARNtest_utilization (BackgroundTasks): inventory collection failed background_task = inventory_collection error = failed to collect inventory: saving inventory to database: Internal Error: unexpected database error: duplicate key value violates unique constraint "inv_caboose_pkey"
64362025-04-30T06:44:02.121ZDEBGtest_utilization (BackgroundTasks): activation complete background_task = inventory_collection elapsed = 509.98941ms iteration = 5 status = {"error":"failed to collect inventory: saving inventory to database: Internal Error: unexpected database error: duplicate key value violates unique constraint \\"inv_caboose_pkey\\""}
64372025-04-30T06:44:02.121ZDEBGtest_utilization (BackgroundTasks): activating background_task = inventory_collection iteration = 6 reason = Signaled
64382025-04-30T06:44:02.121ZTRACtest_utilization (BackgroundTasks): authorize begin action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory
64392025-04-30T06:44:02.121ZTRACtest_utilization (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
64402025-04-30T06:44:02.121ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
64412025-04-30T06:44:02.121ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
64422025-04-30T06:44:02.121ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
64432025-04-30T06:44:02.123ZDEBGtest_utilization: attempt to fetch latest inventory collection result = Ok("found")
64442025-04-30T06:44:02.124ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
64452025-04-30T06:44:02.128ZDEBGtest_utilization (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory result = Ok(())
64462025-04-30T06:44:02.128ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
64472025-04-30T06:44:02.128ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
64482025-04-30T06:44:02.129ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
64492025-04-30T06:44:02.134ZDEBGtest_utilization (BackgroundTasks): inventory_prune_one: nothing eligible for removal (too few) background_task = inventory_collection candidates = [(13d8b8d9-6c6b-47a6-9190-952544edb75b, 0)]
64502025-04-30T06:44:02.134ZTRACtest_utilization (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal
64512025-04-30T06:44:02.135ZTRACtest_utilization (dns): buffer buffer = Length: 67 (0x43) bytes\n0000: 16 41 01 00 00 01 00 00 00 00 00 01 04 5f 6d 67 .A..........._mg\n0010: 73 04 5f 74 63 70 0d 63 6f 6e 74 72 6f 6c 2d 70 s._tcp.control-p\n0020: 6c 61 6e 65 05 6f 78 69 64 65 08 69 6e 74 65 72 lane.oxide.inter\n0030: 6e 61 6c 00 00 21 00 01 00 00 29 04 d0 00 00 00 nal..!....).....\n0040: 00 00 00 ... kind = dns peer_addr = [::1]:53175 req_id = 22284cd3-fd07-4668-b847-5f8112a0526b
64522025-04-30T06:44:02.135ZDEBGtest_utilization (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 5697,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_mgs._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_mgs._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 4,\n 95,\n 109,\n 103,\n 115,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:53175 req_id = 22284cd3-fd07-4668-b847-5f8112a0526b
64532025-04-30T06:44:02.135ZDEBGtest_utilization (store): query key key = _mgs._tcp
64542025-04-30T06:44:02.135ZDEBGtest_utilization (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
64552025-04-30T06:44:02.135ZDEBGtest_utilization (dns): dns response additional_records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal"), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] kind = dns peer_addr = [::1]:53175 query = LowerQuery { name: LowerName(Name("_mgs._tcp.control-plane.oxide.internal.")), original: Query { name: Name("_mgs._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN } } records = [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 48759, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal") })) }] req_id = 22284cd3-fd07-4668-b847-5f8112a0526b
64562025-04-30T06:44:02.136ZDEBGtest_utilization (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgs._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 48759, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }], valid_until: Instant { tv_sec: 4306, tv_nsec: 934947954 } })
64572025-04-30T06:44:02.136ZTRACtest_utilization (DnsResolver): lookup_service_targets: looking up SRV target
64582025-04-30T06:44:02.136ZTRACtest_utilization (dns): buffer buffer = Length: 108 (0x6c) bytes\n0000: 39 d3 01 00 00 01 00 00 00 00 00 01 2d 64 65 6e 9...........-den\n0010: 64 72 69 74 65 2d 62 36 64 36 35 33 34 31 2d 31 drite-b6d65341-1\n0020: 36 37 63 2d 34 31 64 66 2d 39 62 35 63 2d 34 31 67c-41df-9b5c-41\n0030: 63 64 65 64 39 39 63 32 32 39 04 68 6f 73 74 0d cded99c229.host.\n0040: 63 6f 6e 74 72 6f 6c 2d 70 6c 61 6e 65 05 6f 78 control-plane.ox\n0050: 69 64 65 08 69 6e 74 65 72 6e 61 6c 00 00 1c 00 ide.internal....\n0060: 01 00 00 29 04 d0 00 00 00 00 00 00 ...)........ kind = dns peer_addr = [::1]:57012 req_id = b2005486-79fe-413e-b421-4f27b7ad861d
64592025-04-30T06:44:02.136ZDEBGtest_utilization (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 14803,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n query_type: AAAA,\n query_class: IN,\n },\n },\n original: [\n 45,\n 100,\n 101,\n 110,\n 100,\n 114,\n 105,\n 116,\n 101,\n 45,\n 98,\n 54,\n 100,\n 54,\n 53,\n 51,\n 52,\n 49,\n 45,\n 49,\n 54,\n 55,\n 99,\n 45,\n 52,\n 49,\n 100,\n 102,\n 45,\n 57,\n 98,\n 53,\n 99,\n 45,\n 52,\n 49,\n 99,\n 100,\n 101,\n 100,\n 57,\n 57,\n 99,\n 50,\n 50,\n 57,\n 4,\n 104,\n 111,\n 115,\n 116,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 28,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:57012 req_id = b2005486-79fe-413e-b421-4f27b7ad861d
64602025-04-30T06:44:02.136ZDEBGtest_utilization (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
64612025-04-30T06:44:02.136ZDEBGtest_utilization (dns): dns response additional_records = [] kind = dns peer_addr = [::1]:57012 query = LowerQuery { name: LowerName(Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.")), original: Query { name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), query_type: AAAA, query_class: IN } } records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] req_id = b2005486-79fe-413e-b421-4f27b7ad861d
64622025-04-30T06:44:02.206ZTRACtest_utilization (DnsResolver): lookup_all_socket_v6 srv dns_name = _clickhouse-admin-keeper._tcp.control-plane.oxide.internal
64632025-04-30T06:44:02.206ZTRACtest_utilization (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory
64642025-04-30T06:44:02.206ZTRACtest_utilization (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
64652025-04-30T06:44:02.206ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
64662025-04-30T06:44:02.206ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
64672025-04-30T06:44:02.207ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
64682025-04-30T06:44:02.211ZTRACtest_utilization (dns): buffer buffer = Length: 87 (0x57) bytes\n0000: 85 35 01 00 00 01 00 00 00 00 00 01 18 5f 63 6c .5..........._cl\n0010: 69 63 6b 68 6f 75 73 65 2d 61 64 6d 69 6e 2d 6b ickhouse-admin-k\n0020: 65 65 70 65 72 04 5f 74 63 70 0d 63 6f 6e 74 72 eeper._tcp.contr\n0030: 6f 6c 2d 70 6c 61 6e 65 05 6f 78 69 64 65 08 69 ol-plane.oxide.i\n0040: 6e 74 65 72 6e 61 6c 00 00 21 00 01 00 00 29 04 nternal..!....).\n0050: d0 00 00 00 00 00 00 ....... kind = dns peer_addr = [::1]:56539 req_id = cebf18fe-076c-47a0-af70-9944cc21d7a3
64692025-04-30T06:44:02.211ZDEBGtest_utilization (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 34101,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 24,\n 95,\n 99,\n 108,\n 105,\n 99,\n 107,\n 104,\n 111,\n 117,\n 115,\n 101,\n 45,\n 97,\n 100,\n 109,\n 105,\n 110,\n 45,\n 107,\n 101,\n 101,\n 112,\n 101,\n 114,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:56539 req_id = cebf18fe-076c-47a0-af70-9944cc21d7a3
64702025-04-30T06:44:02.211ZDEBGtest_utilization (store): query key key = _clickhouse-admin-keeper._tcp
64712025-04-30T06:44:02.212ZERROtest_utilization (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 34101, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 1, }, query: WireQuery { query: LowerQuery { name: LowerName( Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), ), original: Query { name: Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN, }, }, original: [ 24, 95, 99, 108, 105, 99, 107, 104, 111, 117, 115, 101, 45, 97, 100, 109, 105, 110, 45, 107, 101, 101, 112, 101, 114, 4, 95, 116, 99, 112, 13, 99, 111, 110, 116, 114, 111, 108, 45, 112, 108, 97, 110, 101, 5, 111, 120, 105, 100, 101, 8, 105, 110, 116, 101, 114, 110, 97, 108, 0, 0, 33, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: Some( Edns { rcode_high: 0, version: 0, dnssec_ok: false, max_payload: 1232, options: OPT { options: {}, }, }, ), } NXDOMAIN: no records found for name: "_clickhouse-admin-keeper._tcp.control-plane.oxide.internal." kind = dns peer_addr = [::1]:56539 req_id = cebf18fe-076c-47a0-af70-9944cc21d7a3
64722025-04-30T06:44:02.213ZDEBGtest_utilization (BackgroundTasks): begin collection background_task = inventory_collection
64732025-04-30T06:44:02.213ZDEBGtest_utilization (BackgroundTasks): begin collection from MGS background_task = inventory_collection mgs_url = http://[::1]:48759
64742025-04-30T06:44:02.213ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/ignition
64752025-04-30T06:44:02.213ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
64762025-04-30T06:44:02.233ZDEBGtest_utilization (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory result = Ok(())
64772025-04-30T06:44:02.233ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
64782025-04-30T06:44:02.233ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
64792025-04-30T06:44:02.233ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
64802025-04-30T06:44:02.239ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
64812025-04-30T06:44:02.239ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
64822025-04-30T06:44:02.240ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
64832025-04-30T06:44:02.240ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/ignition", status: 200, headers: {"content-type": "application/json", "x-request-id": "a889d1e5-5285-4bf2-b7f3-597f5cb839e7", "content-length": "835", "date": "Wed, 30 Apr 2025 06:44:02 GMT"} })
64842025-04-30T06:44:02.241ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0
64852025-04-30T06:44:02.244ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "0174618c-7832-4923-ae8c-7ccffe99ebab", "content-length": "734", "date": "Wed, 30 Apr 2025 06:44:02 GMT"} })
64862025-04-30T06:44:02.244ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0/component/sp/caboose?firmware_slot=0
64872025-04-30T06:44:02.248ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "6798a92b-5309-4478-85b4-a0cdffc50636", "content-length": "111", "date": "Wed, 30 Apr 2025 06:44:02 GMT"} })
64882025-04-30T06:44:02.248ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0/component/sp/caboose?firmware_slot=1
64892025-04-30T06:44:02.256ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "b9db8a6d-8fe3-468b-a9c2-e7126fa53bb0", "content-length": "111", "date": "Wed, 30 Apr 2025 06:44:02 GMT"} })
64902025-04-30T06:44:02.256ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0/component/rot/caboose?firmware_slot=0
64912025-04-30T06:44:02.263ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "4536c217-87ec-4ccd-8055-81c2e3a3c982", "content-length": "170", "date": "Wed, 30 Apr 2025 06:44:02 GMT"} })
64922025-04-30T06:44:02.263ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0/component/rot/caboose?firmware_slot=1
64932025-04-30T06:44:02.281ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "ec55f190-5103-47c9-ba6d-b94126663ad1", "content-length": "170", "date": "Wed, 30 Apr 2025 06:44:02 GMT"} })
64942025-04-30T06:44:02.281ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0/component/stage0/caboose?firmware_slot=0
64952025-04-30T06:44:02.284ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "111cd5f5-bac2-4dab-bd37-bbbf5fd62a2e", "content-length": "179", "date": "Wed, 30 Apr 2025 06:44:02 GMT"} })
64962025-04-30T06:44:02.284ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0/component/stage0/caboose?firmware_slot=1
64972025-04-30T06:44:02.291ZDEBGtest_utilization: attempt to fetch latest inventory collection result = Ok("found")
64982025-04-30T06:44:02.317ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "9fd47dd3-7277-4177-9cae-f1ba77569e60", "content-length": "179", "date": "Wed, 30 Apr 2025 06:44:02 GMT"} })
64992025-04-30T06:44:02.317ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0/component/rot/cmpa
65002025-04-30T06:44:02.318ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "bb822b97-8e2b-4226-a3ec-e37a68fa173f", "content-length": "702", "date": "Wed, 30 Apr 2025 06:44:02 GMT"} })
65012025-04-30T06:44:02.318ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0/component/rot/cfpa
65022025-04-30T06:44:02.319ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "d729668b-8628-41e1-845c-180a073cb471", "content-length": "727", "date": "Wed, 30 Apr 2025 06:44:02 GMT"} })
65032025-04-30T06:44:02.319ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0/component/rot/cfpa
65042025-04-30T06:44:02.321ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "391ab73f-992e-4901-a79a-542f6b52d2dc", "content-length": "729", "date": "Wed, 30 Apr 2025 06:44:02 GMT"} })
65052025-04-30T06:44:02.321ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0/component/rot/cfpa
65062025-04-30T06:44:02.322ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "cf585d4c-993d-44b2-8ea9-73325040b7da", "content-length": "728", "date": "Wed, 30 Apr 2025 06:44:02 GMT"} })
65072025-04-30T06:44:02.323ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1
65082025-04-30T06:44:02.324ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1", status: 200, headers: {"content-type": "application/json", "x-request-id": "7934ebbe-14e1-489d-9721-91a45d36d07f", "content-length": "734", "date": "Wed, 30 Apr 2025 06:44:02 GMT"} })
65092025-04-30T06:44:02.324ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1/component/sp/caboose?firmware_slot=0
65102025-04-30T06:44:02.327ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "e908ce34-d83f-4681-af4e-2720f353aa6b", "content-length": "111", "date": "Wed, 30 Apr 2025 06:44:02 GMT"} })
65112025-04-30T06:44:02.327ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1/component/sp/caboose?firmware_slot=1
65122025-04-30T06:44:02.331ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "7af2ca93-f62d-42e6-a5cd-8f87103209f7", "content-length": "111", "date": "Wed, 30 Apr 2025 06:44:02 GMT"} })
65132025-04-30T06:44:02.331ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1/component/rot/caboose?firmware_slot=0
65142025-04-30T06:44:02.334ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "ae3dfcca-919d-4624-b5ab-f43081fae7dc", "content-length": "170", "date": "Wed, 30 Apr 2025 06:44:02 GMT"} })
65152025-04-30T06:44:02.334ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1/component/rot/caboose?firmware_slot=1
65162025-04-30T06:44:02.336ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "5912ed2b-168c-49d2-90af-3e0cfe5f74c5", "content-length": "170", "date": "Wed, 30 Apr 2025 06:44:02 GMT"} })
65172025-04-30T06:44:02.336ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1/component/stage0/caboose?firmware_slot=0
65182025-04-30T06:44:02.339ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "3c0e96fb-d4d6-4a69-940e-d194006eca64", "content-length": "179", "date": "Wed, 30 Apr 2025 06:44:02 GMT"} })
65192025-04-30T06:44:02.339ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1/component/stage0/caboose?firmware_slot=1
65202025-04-30T06:44:02.341ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "aa4fe501-5ce2-4b60-a810-466e836d390b", "content-length": "179", "date": "Wed, 30 Apr 2025 06:44:02 GMT"} })
65212025-04-30T06:44:02.341ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1/component/rot/cmpa
65222025-04-30T06:44:02.342ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "482897e7-29dd-473c-9994-5d5e6aea933e", "content-length": "702", "date": "Wed, 30 Apr 2025 06:44:02 GMT"} })
65232025-04-30T06:44:02.342ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1/component/rot/cfpa
65242025-04-30T06:44:02.342ZTRACtest_utilization (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory
65252025-04-30T06:44:02.342ZTRACtest_utilization (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
65262025-04-30T06:44:02.342ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
65272025-04-30T06:44:02.342ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
65282025-04-30T06:44:02.343ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
65292025-04-30T06:44:02.344ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "6d24e421-beb2-42f4-ab6b-aa8ddaafc45f", "content-length": "727", "date": "Wed, 30 Apr 2025 06:44:02 GMT"} })
65302025-04-30T06:44:02.344ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1/component/rot/cfpa
65312025-04-30T06:44:02.346ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "d233409a-b2b3-4000-a417-db46972cab90", "content-length": "729", "date": "Wed, 30 Apr 2025 06:44:02 GMT"} })
65322025-04-30T06:44:02.346ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1/component/rot/cfpa
65332025-04-30T06:44:02.346ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
65342025-04-30T06:44:02.354ZDEBGtest_utilization (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory result = Ok(())
65352025-04-30T06:44:02.354ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
65362025-04-30T06:44:02.354ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
65372025-04-30T06:44:02.354ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
65382025-04-30T06:44:02.357ZDEBGtest_utilization (results-sink): inserting 1 samples into database collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
65392025-04-30T06:44:02.357ZDEBGtest_utilization (results-sink-cluster): inserting 1 samples into database collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
65402025-04-30T06:44:02.357ZDEBGtest_utilization (clickhouse-client): unrolling 1 total samples collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 4d1485f0-110e-4161-9a33-c4610d68be5d
65412025-04-30T06:44:02.358ZTRACtest_utilization (clickhouse-client): inserting data collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 4d1485f0-110e-4161-9a33-c4610d68be5d n_columns = 4 n_rows = 2 sql = INSERT INTO oximeter.fields_string FORMAT Native
65422025-04-30T06:44:02.358ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
65432025-04-30T06:44:02.358ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
65442025-04-30T06:44:02.359ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
65452025-04-30T06:44:02.359ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "78b61c3b-3d7a-446d-bdfb-c67167f5745f", "content-length": "728", "date": "Wed, 30 Apr 2025 06:44:02 GMT"} })
65462025-04-30T06:44:02.359ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0
65472025-04-30T06:44:02.361ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "d48326e6-b108-4c2a-9346-54decf095e7c", "content-length": "723", "date": "Wed, 30 Apr 2025 06:44:02 GMT"} })
65482025-04-30T06:44:02.361ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0/component/sp/caboose?firmware_slot=0
65492025-04-30T06:44:02.362ZDEBGtest_utilization (clickhouse-client): inserted rows into table collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 4d1485f0-110e-4161-9a33-c4610d68be5d n_rows = 2 table_name = fields_string
65502025-04-30T06:44:02.362ZTRACtest_utilization (clickhouse-client): inserting data collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 4d1485f0-110e-4161-9a33-c4610d68be5d n_columns = 4 n_rows = 1 sql = INSERT INTO oximeter.measurements_i64 FORMAT Native
65512025-04-30T06:44:02.365ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "87d9d1d3-4ce4-4cd5-ba7f-97f779a6eef4", "content-length": "109", "date": "Wed, 30 Apr 2025 06:44:02 GMT"} })
65522025-04-30T06:44:02.365ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0/component/sp/caboose?firmware_slot=1
65532025-04-30T06:44:02.365ZDEBGtest_utilization (clickhouse-client): inserted rows into table collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 4d1485f0-110e-4161-9a33-c4610d68be5d n_rows = 1 table_name = measurements_i64
65542025-04-30T06:44:02.365ZTRACtest_utilization (results-sink): successfully inserted samples collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
65552025-04-30T06:44:02.368ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "35bdad98-a01f-44c4-901a-dab55ead144a", "content-length": "109", "date": "Wed, 30 Apr 2025 06:44:02 GMT"} })
65562025-04-30T06:44:02.368ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0/component/rot/caboose?firmware_slot=0
65572025-04-30T06:44:02.371ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "8d3fa029-f73f-4c16-93b6-ee7bb7338782", "content-length": "169", "date": "Wed, 30 Apr 2025 06:44:02 GMT"} })
65582025-04-30T06:44:02.372ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0/component/rot/caboose?firmware_slot=1
65592025-04-30T06:44:02.381ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "48d31e39-1773-46bd-909d-5256bce2fe75", "content-length": "169", "date": "Wed, 30 Apr 2025 06:44:02 GMT"} })
65602025-04-30T06:44:02.381ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0/component/stage0/caboose?firmware_slot=0
65612025-04-30T06:44:02.383ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "842fe2d8-81a2-4b83-a17e-094e1b24cca6", "content-length": "178", "date": "Wed, 30 Apr 2025 06:44:02 GMT"} })
65622025-04-30T06:44:02.383ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0/component/stage0/caboose?firmware_slot=1
65632025-04-30T06:44:02.385ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "6ea00f26-b54e-46e1-b078-8bdaa9668c1c", "content-length": "178", "date": "Wed, 30 Apr 2025 06:44:02 GMT"} })
65642025-04-30T06:44:02.385ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0/component/rot/cmpa
65652025-04-30T06:44:02.386ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "225e2cb4-b426-4e8d-a0cf-1372690ca370", "content-length": "702", "date": "Wed, 30 Apr 2025 06:44:02 GMT"} })
65662025-04-30T06:44:02.387ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0/component/rot/cfpa
65672025-04-30T06:44:02.388ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "b6348b97-74e0-4ef5-b401-514b8861fb6e", "content-length": "727", "date": "Wed, 30 Apr 2025 06:44:02 GMT"} })
65682025-04-30T06:44:02.388ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0/component/rot/cfpa
65692025-04-30T06:44:02.390ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "61bfe968-b5da-4444-8dba-5441f2f2fb3d", "content-length": "729", "date": "Wed, 30 Apr 2025 06:44:02 GMT"} })
65702025-04-30T06:44:02.390ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0/component/rot/cfpa
65712025-04-30T06:44:02.393ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "a69f7744-1264-4138-b167-6855f1ebde87", "content-length": "728", "date": "Wed, 30 Apr 2025 06:44:02 GMT"} })
65722025-04-30T06:44:02.393ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1
65732025-04-30T06:44:02.394ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1", status: 200, headers: {"content-type": "application/json", "x-request-id": "0fee97a8-0693-42c5-a6e8-0607bbeaf414", "content-length": "723", "date": "Wed, 30 Apr 2025 06:44:02 GMT"} })
65742025-04-30T06:44:02.395ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1/component/sp/caboose?firmware_slot=0
65752025-04-30T06:44:02.398ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "5603985c-1259-4302-9369-dca2457e5985", "content-length": "109", "date": "Wed, 30 Apr 2025 06:44:02 GMT"} })
65762025-04-30T06:44:02.398ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1/component/sp/caboose?firmware_slot=1
65772025-04-30T06:44:02.400ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "cc44e0b3-e022-48f3-aaa8-9c4805febbc8", "content-length": "109", "date": "Wed, 30 Apr 2025 06:44:02 GMT"} })
65782025-04-30T06:44:02.400ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1/component/rot/caboose?firmware_slot=0
65792025-04-30T06:44:02.403ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "a4977cb4-614e-47ad-8440-ab77fe28708a", "content-length": "169", "date": "Wed, 30 Apr 2025 06:44:02 GMT"} })
65802025-04-30T06:44:02.403ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1/component/rot/caboose?firmware_slot=1
65812025-04-30T06:44:02.405ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "84553d54-0f6f-4ee9-8c3a-f8f88b1ee099", "content-length": "169", "date": "Wed, 30 Apr 2025 06:44:02 GMT"} })
65822025-04-30T06:44:02.405ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1/component/stage0/caboose?firmware_slot=0
65832025-04-30T06:44:02.408ZDEBGtest_utilization: attempt to fetch latest inventory collection result = Ok("found")
65842025-04-30T06:44:02.410ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "cea96a4c-beb9-4ee1-9294-c7da8951278c", "content-length": "178", "date": "Wed, 30 Apr 2025 06:44:02 GMT"} })
65852025-04-30T06:44:02.410ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1/component/stage0/caboose?firmware_slot=1
65862025-04-30T06:44:02.412ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "37df88b3-dd26-4d74-b40b-54cfcb265c77", "content-length": "178", "date": "Wed, 30 Apr 2025 06:44:02 GMT"} })
65872025-04-30T06:44:02.413ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1/component/rot/cmpa
65882025-04-30T06:44:02.413ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "3f509d0f-c42b-4fd1-867f-e43c0048694e", "content-length": "702", "date": "Wed, 30 Apr 2025 06:44:02 GMT"} })
65892025-04-30T06:44:02.413ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1/component/rot/cfpa
65902025-04-30T06:44:02.414ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "88902149-f19e-464a-88e7-efb9abb6c4a0", "content-length": "727", "date": "Wed, 30 Apr 2025 06:44:02 GMT"} })
65912025-04-30T06:44:02.414ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1/component/rot/cfpa
65922025-04-30T06:44:02.415ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "fcda6474-b085-4652-96c5-11290989e638", "content-length": "729", "date": "Wed, 30 Apr 2025 06:44:02 GMT"} })
65932025-04-30T06:44:02.416ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1/component/rot/cfpa
65942025-04-30T06:44:02.417ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "1a80faac-e1a0-456f-a035-7167bb4918a5", "content-length": "728", "date": "Wed, 30 Apr 2025 06:44:02 GMT"} })
65952025-04-30T06:44:02.417ZTRACtest_utilization (BackgroundTasks): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet
65962025-04-30T06:44:02.417ZTRACtest_utilization (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
65972025-04-30T06:44:02.417ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
65982025-04-30T06:44:02.417ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
65992025-04-30T06:44:02.417ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
66002025-04-30T06:44:02.426ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
66012025-04-30T06:44:02.433ZDEBGtest_utilization (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet result = Ok(())
66022025-04-30T06:44:02.433ZTRACtest_utilization (BackgroundTasks): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet
66032025-04-30T06:44:02.433ZTRACtest_utilization (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
66042025-04-30T06:44:02.433ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
66052025-04-30T06:44:02.433ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
66062025-04-30T06:44:02.434ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
66072025-04-30T06:44:02.444ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
66082025-04-30T06:44:02.448ZDEBGtest_utilization (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet result = Ok(())
66092025-04-30T06:44:02.448ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
66102025-04-30T06:44:02.448ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
66112025-04-30T06:44:02.449ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
66122025-04-30T06:44:02.530ZDEBGtest_utilization (BackgroundTasks): begin collection from Sled Agent background_task = inventory_collection sled_agent_url = http://[::1]:39365
66132025-04-30T06:44:02.530ZDEBGtest_utilization (BackgroundTasks): client request SledAgent = http://[::1]:39365 background_task = inventory_collection body = None method = GET uri = http://[::1]:39365/inventory
66142025-04-30T06:44:02.532ZTRACtest_utilization (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory
66152025-04-30T06:44:02.532ZTRACtest_utilization (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
66162025-04-30T06:44:02.532ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
66172025-04-30T06:44:02.532ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
66182025-04-30T06:44:02.534ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
66192025-04-30T06:44:02.534ZINFOtest_utilization (dropshot): accepted connection local_addr = [::1]:39365 remote_addr = [::1]:40139 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
66202025-04-30T06:44:02.534ZWARNtest_utilization (results-sink-cluster): failed to insert some results into metric DB collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 error = Failed to check out connection to database: No backends found for this service file = oximeter/collector/src/results_sink.rs:92
66212025-04-30T06:44:02.545ZTRACtest_utilization (dropshot): incoming request local_addr = [::1]:39365 method = GET remote_addr = [::1]:40139 req_id = 96d329f2-f684-4894-b958-8ac8432eb50d sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /inventory
66222025-04-30T06:44:02.545ZINFOtest_utilization (dropshot): request completed latency_us = 568 local_addr = [::1]:39365 method = GET remote_addr = [::1]:40139 req_id = 96d329f2-f684-4894-b958-8ac8432eb50d response_code = 200 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /inventory
66232025-04-30T06:44:02.546ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
66242025-04-30T06:44:02.555ZDEBGtest_utilization (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory result = Ok(())
66252025-04-30T06:44:02.555ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
66262025-04-30T06:44:02.555ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
66272025-04-30T06:44:02.556ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
66282025-04-30T06:44:02.557ZDEBGtest_utilization (BackgroundTasks): client response SledAgent = http://[::1]:39365 background_task = inventory_collection result = Ok(Response { url: "http://[::1]:39365/inventory", status: 200, headers: {"content-type": "application/json", "x-request-id": "96d329f2-f684-4894-b958-8ac8432eb50d", "content-length": "3141", "date": "Wed, 30 Apr 2025 06:44:02 GMT"} })
66292025-04-30T06:44:02.559ZDEBGtest_utilization (BackgroundTasks): begin collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
66302025-04-30T06:44:02.560ZDEBGtest_utilization (BackgroundTasks): end collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
66312025-04-30T06:44:02.560ZDEBGtest_utilization (BackgroundTasks): finished collection background_task = inventory_collection
66322025-04-30T06:44:02.568ZTRACtest_utilization (BackgroundTasks): authorize begin action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory
66332025-04-30T06:44:02.568ZTRACtest_utilization (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
66342025-04-30T06:44:02.568ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
66352025-04-30T06:44:02.568ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
66362025-04-30T06:44:02.569ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
66372025-04-30T06:44:02.575ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
66382025-04-30T06:44:02.580ZDEBGtest_utilization (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory result = Ok(())
66392025-04-30T06:44:02.580ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
66402025-04-30T06:44:02.580ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
66412025-04-30T06:44:02.581ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
66422025-04-30T06:44:02.582ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
66432025-04-30T06:44:02.582ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
66442025-04-30T06:44:02.583ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
66452025-04-30T06:44:02.637ZWARNtest_utilization (datastore): Non-retryable transaction failure err = DatabaseError(UniqueViolation, "duplicate key value violates unique constraint \\"inv_caboose_pkey\\"") transaction = inventory_insert_collection
66462025-04-30T06:44:02.637ZWARNtest_utilization (BackgroundTasks): inventory collection failed background_task = inventory_collection error = failed to collect inventory: saving inventory to database: Internal Error: unexpected database error: duplicate key value violates unique constraint "inv_caboose_pkey"
66472025-04-30T06:44:02.648ZDEBGtest_utilization (BackgroundTasks): activation complete background_task = inventory_collection elapsed = 527.489078ms iteration = 6 status = {"error":"failed to collect inventory: saving inventory to database: Internal Error: unexpected database error: duplicate key value violates unique constraint \\"inv_caboose_pkey\\""}
66482025-04-30T06:44:02.648ZDEBGtest_utilization (BackgroundTasks): activating background_task = inventory_collection iteration = 7 reason = Signaled
66492025-04-30T06:44:02.648ZTRACtest_utilization (BackgroundTasks): authorize begin action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory
66502025-04-30T06:44:02.648ZTRACtest_utilization (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
66512025-04-30T06:44:02.649ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
66522025-04-30T06:44:02.649ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
66532025-04-30T06:44:02.652ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
66542025-04-30T06:44:02.656ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
66552025-04-30T06:44:02.661ZDEBGtest_utilization (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory result = Ok(())
66562025-04-30T06:44:02.661ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
66572025-04-30T06:44:02.661ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
66582025-04-30T06:44:02.662ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
66592025-04-30T06:44:02.669ZDEBGtest_utilization (BackgroundTasks): inventory_prune_one: nothing eligible for removal (too few) background_task = inventory_collection candidates = [(13d8b8d9-6c6b-47a6-9190-952544edb75b, 0)]
66602025-04-30T06:44:02.670ZTRACtest_utilization (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal
66612025-04-30T06:44:02.671ZDEBGtest_utilization: attempt to fetch latest inventory collection result = Ok("found")
66622025-04-30T06:44:02.672ZTRACtest_utilization (dns): buffer buffer = Length: 67 (0x43) bytes\n0000: e5 9d 01 00 00 01 00 00 00 00 00 01 04 5f 6d 67 ............._mg\n0010: 73 04 5f 74 63 70 0d 63 6f 6e 74 72 6f 6c 2d 70 s._tcp.control-p\n0020: 6c 61 6e 65 05 6f 78 69 64 65 08 69 6e 74 65 72 lane.oxide.inter\n0030: 6e 61 6c 00 00 21 00 01 00 00 29 04 d0 00 00 00 nal..!....).....\n0040: 00 00 00 ... kind = dns peer_addr = [::1]:59189 req_id = efa828f3-f75b-4059-a669-057fdcf81190
66632025-04-30T06:44:02.672ZDEBGtest_utilization (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 58781,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_mgs._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_mgs._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 4,\n 95,\n 109,\n 103,\n 115,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:59189 req_id = efa828f3-f75b-4059-a669-057fdcf81190
66642025-04-30T06:44:02.672ZDEBGtest_utilization (store): query key key = _mgs._tcp
66652025-04-30T06:44:02.673ZDEBGtest_utilization (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
66662025-04-30T06:44:02.673ZDEBGtest_utilization (dns): dns response additional_records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal"), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] kind = dns peer_addr = [::1]:59189 query = LowerQuery { name: LowerName(Name("_mgs._tcp.control-plane.oxide.internal.")), original: Query { name: Name("_mgs._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN } } records = [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 48759, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal") })) }] req_id = efa828f3-f75b-4059-a669-057fdcf81190
66672025-04-30T06:44:02.673ZDEBGtest_utilization (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgs._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 48759, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }], valid_until: Instant { tv_sec: 4307, tv_nsec: 472806112 } })
66682025-04-30T06:44:02.674ZTRACtest_utilization (DnsResolver): lookup_service_targets: looking up SRV target
66692025-04-30T06:44:02.674ZTRACtest_utilization (dns): buffer buffer = Length: 108 (0x6c) bytes\n0000: 80 9b 01 00 00 01 00 00 00 00 00 01 2d 64 65 6e ............-den\n0010: 64 72 69 74 65 2d 62 36 64 36 35 33 34 31 2d 31 drite-b6d65341-1\n0020: 36 37 63 2d 34 31 64 66 2d 39 62 35 63 2d 34 31 67c-41df-9b5c-41\n0030: 63 64 65 64 39 39 63 32 32 39 04 68 6f 73 74 0d cded99c229.host.\n0040: 63 6f 6e 74 72 6f 6c 2d 70 6c 61 6e 65 05 6f 78 control-plane.ox\n0050: 69 64 65 08 69 6e 74 65 72 6e 61 6c 00 00 1c 00 ide.internal....\n0060: 01 00 00 29 04 d0 00 00 00 00 00 00 ...)........ kind = dns peer_addr = [::1]:63080 req_id = b1088a58-7547-47c5-9d37-195ff58b1ea6
66702025-04-30T06:44:02.674ZDEBGtest_utilization (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 32923,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n query_type: AAAA,\n query_class: IN,\n },\n },\n original: [\n 45,\n 100,\n 101,\n 110,\n 100,\n 114,\n 105,\n 116,\n 101,\n 45,\n 98,\n 54,\n 100,\n 54,\n 53,\n 51,\n 52,\n 49,\n 45,\n 49,\n 54,\n 55,\n 99,\n 45,\n 52,\n 49,\n 100,\n 102,\n 45,\n 57,\n 98,\n 53,\n 99,\n 45,\n 52,\n 49,\n 99,\n 100,\n 101,\n 100,\n 57,\n 57,\n 99,\n 50,\n 50,\n 57,\n 4,\n 104,\n 111,\n 115,\n 116,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 28,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:63080 req_id = b1088a58-7547-47c5-9d37-195ff58b1ea6
66712025-04-30T06:44:02.675ZDEBGtest_utilization (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
66722025-04-30T06:44:02.675ZDEBGtest_utilization (dns): dns response additional_records = [] kind = dns peer_addr = [::1]:63080 query = LowerQuery { name: LowerName(Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.")), original: Query { name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), query_type: AAAA, query_class: IN } } records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] req_id = b1088a58-7547-47c5-9d37-195ff58b1ea6
66732025-04-30T06:44:02.775ZTRACtest_utilization (DnsResolver): lookup_all_socket_v6 srv dns_name = _clickhouse-admin-keeper._tcp.control-plane.oxide.internal
66742025-04-30T06:44:02.776ZTRACtest_utilization (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory
66752025-04-30T06:44:02.776ZTRACtest_utilization (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
66762025-04-30T06:44:02.776ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
66772025-04-30T06:44:02.776ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
66782025-04-30T06:44:02.777ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
66792025-04-30T06:44:02.778ZTRACtest_utilization (dns): buffer buffer = Length: 87 (0x57) bytes\n0000: ad 2f 01 00 00 01 00 00 00 00 00 01 18 5f 63 6c ./..........._cl\n0010: 69 63 6b 68 6f 75 73 65 2d 61 64 6d 69 6e 2d 6b ickhouse-admin-k\n0020: 65 65 70 65 72 04 5f 74 63 70 0d 63 6f 6e 74 72 eeper._tcp.contr\n0030: 6f 6c 2d 70 6c 61 6e 65 05 6f 78 69 64 65 08 69 ol-plane.oxide.i\n0040: 6e 74 65 72 6e 61 6c 00 00 21 00 01 00 00 29 04 nternal..!....).\n0050: d0 00 00 00 00 00 00 ....... kind = dns peer_addr = [::1]:64983 req_id = 8bcd0b2b-2c22-4bb1-ba22-754af97d3a76
66802025-04-30T06:44:02.778ZDEBGtest_utilization (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 44335,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 24,\n 95,\n 99,\n 108,\n 105,\n 99,\n 107,\n 104,\n 111,\n 117,\n 115,\n 101,\n 45,\n 97,\n 100,\n 109,\n 105,\n 110,\n 45,\n 107,\n 101,\n 101,\n 112,\n 101,\n 114,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:64983 req_id = 8bcd0b2b-2c22-4bb1-ba22-754af97d3a76
66812025-04-30T06:44:02.778ZDEBGtest_utilization (store): query key key = _clickhouse-admin-keeper._tcp
66822025-04-30T06:44:02.778ZERROtest_utilization (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 44335, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 1, }, query: WireQuery { query: LowerQuery { name: LowerName( Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), ), original: Query { name: Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN, }, }, original: [ 24, 95, 99, 108, 105, 99, 107, 104, 111, 117, 115, 101, 45, 97, 100, 109, 105, 110, 45, 107, 101, 101, 112, 101, 114, 4, 95, 116, 99, 112, 13, 99, 111, 110, 116, 114, 111, 108, 45, 112, 108, 97, 110, 101, 5, 111, 120, 105, 100, 101, 8, 105, 110, 116, 101, 114, 110, 97, 108, 0, 0, 33, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: Some( Edns { rcode_high: 0, version: 0, dnssec_ok: false, max_payload: 1232, options: OPT { options: {}, }, }, ), } NXDOMAIN: no records found for name: "_clickhouse-admin-keeper._tcp.control-plane.oxide.internal." kind = dns peer_addr = [::1]:64983 req_id = 8bcd0b2b-2c22-4bb1-ba22-754af97d3a76
66832025-04-30T06:44:02.778ZDEBGtest_utilization (BackgroundTasks): begin collection background_task = inventory_collection
66842025-04-30T06:44:02.778ZDEBGtest_utilization (BackgroundTasks): begin collection from MGS background_task = inventory_collection mgs_url = http://[::1]:48759
66852025-04-30T06:44:02.778ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/ignition
66862025-04-30T06:44:02.780ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
66872025-04-30T06:44:02.786ZDEBGtest_utilization (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory result = Ok(())
66882025-04-30T06:44:02.786ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
66892025-04-30T06:44:02.786ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
66902025-04-30T06:44:02.787ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
66912025-04-30T06:44:02.789ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/ignition", status: 200, headers: {"content-type": "application/json", "x-request-id": "b1e0abd9-96a9-4f9d-927d-9d6c094ac3d5", "content-length": "835", "date": "Wed, 30 Apr 2025 06:44:02 GMT"} })
66922025-04-30T06:44:02.789ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0
66932025-04-30T06:44:02.790ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
66942025-04-30T06:44:02.790ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
66952025-04-30T06:44:02.791ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
66962025-04-30T06:44:02.792ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "f5c5bc02-07d4-459e-8ccd-fe52207961c8", "content-length": "734", "date": "Wed, 30 Apr 2025 06:44:02 GMT"} })
66972025-04-30T06:44:02.792ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0/component/sp/caboose?firmware_slot=0
66982025-04-30T06:44:02.795ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "7ebaa052-dafc-4217-8c6e-cf5ab6621552", "content-length": "111", "date": "Wed, 30 Apr 2025 06:44:02 GMT"} })
66992025-04-30T06:44:02.795ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0/component/sp/caboose?firmware_slot=1
67002025-04-30T06:44:02.797ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "406c1e68-756d-4e61-965f-d2f3ab12b1f4", "content-length": "111", "date": "Wed, 30 Apr 2025 06:44:02 GMT"} })
67012025-04-30T06:44:02.797ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0/component/rot/caboose?firmware_slot=0
67022025-04-30T06:44:02.805ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "a74c1f9a-9893-48a6-ac4f-233e40655e38", "content-length": "170", "date": "Wed, 30 Apr 2025 06:44:02 GMT"} })
67032025-04-30T06:44:02.805ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0/component/rot/caboose?firmware_slot=1
67042025-04-30T06:44:02.807ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "c1a7b5aa-ba1d-43a3-91a8-a0e84acf4b3f", "content-length": "170", "date": "Wed, 30 Apr 2025 06:44:02 GMT"} })
67052025-04-30T06:44:02.808ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0/component/stage0/caboose?firmware_slot=0
67062025-04-30T06:44:02.812ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "57ab0138-238c-4fa7-a621-111e39d9ad8d", "content-length": "179", "date": "Wed, 30 Apr 2025 06:44:02 GMT"} })
67072025-04-30T06:44:02.812ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0/component/stage0/caboose?firmware_slot=1
67082025-04-30T06:44:02.814ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "5b395ba4-bf3a-43b6-80b4-8335689daa49", "content-length": "179", "date": "Wed, 30 Apr 2025 06:44:02 GMT"} })
67092025-04-30T06:44:02.814ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0/component/rot/cmpa
67102025-04-30T06:44:02.815ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "98862b6a-4c0c-4d6b-8cda-c741c65e153f", "content-length": "702", "date": "Wed, 30 Apr 2025 06:44:02 GMT"} })
67112025-04-30T06:44:02.815ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0/component/rot/cfpa
67122025-04-30T06:44:02.817ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "88ae7a2b-df0c-4186-ac0d-4b213f6c2e71", "content-length": "727", "date": "Wed, 30 Apr 2025 06:44:02 GMT"} })
67132025-04-30T06:44:02.817ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0/component/rot/cfpa
67142025-04-30T06:44:02.818ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "32e92073-9359-4c71-b48d-d8a557defd29", "content-length": "729", "date": "Wed, 30 Apr 2025 06:44:02 GMT"} })
67152025-04-30T06:44:02.818ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0/component/rot/cfpa
67162025-04-30T06:44:02.820ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "77827ad8-5f3f-4c9d-9056-50c8188a4ee7", "content-length": "728", "date": "Wed, 30 Apr 2025 06:44:02 GMT"} })
67172025-04-30T06:44:02.820ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1
67182025-04-30T06:44:02.822ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1", status: 200, headers: {"content-type": "application/json", "x-request-id": "eab1a1c9-9d2a-44b7-9b89-13f2d6db940b", "content-length": "734", "date": "Wed, 30 Apr 2025 06:44:02 GMT"} })
67192025-04-30T06:44:02.822ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1/component/sp/caboose?firmware_slot=0
67202025-04-30T06:44:02.828ZDEBGtest_utilization: attempt to fetch latest inventory collection result = Ok("found")
67212025-04-30T06:44:02.830ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "62c51f47-9bfe-45eb-9175-1661af9a87b9", "content-length": "111", "date": "Wed, 30 Apr 2025 06:44:02 GMT"} })
67222025-04-30T06:44:02.830ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1/component/sp/caboose?firmware_slot=1
67232025-04-30T06:44:02.833ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "8d53c875-c13c-447b-a20c-08ba34e6d8e3", "content-length": "111", "date": "Wed, 30 Apr 2025 06:44:02 GMT"} })
67242025-04-30T06:44:02.833ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1/component/rot/caboose?firmware_slot=0
67252025-04-30T06:44:02.837ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "022f1ad8-7cd8-4ed1-96c9-86e33969af2e", "content-length": "170", "date": "Wed, 30 Apr 2025 06:44:02 GMT"} })
67262025-04-30T06:44:02.837ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1/component/rot/caboose?firmware_slot=1
67272025-04-30T06:44:02.839ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "051a99a0-1588-4c80-9317-05ec35ff59d3", "content-length": "170", "date": "Wed, 30 Apr 2025 06:44:02 GMT"} })
67282025-04-30T06:44:02.840ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1/component/stage0/caboose?firmware_slot=0
67292025-04-30T06:44:02.842ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "42106f14-866e-409c-b70a-ff703930e487", "content-length": "179", "date": "Wed, 30 Apr 2025 06:44:02 GMT"} })
67302025-04-30T06:44:02.842ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1/component/stage0/caboose?firmware_slot=1
67312025-04-30T06:44:02.846ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "2f28cbdc-7d96-4375-aa2f-7c72b3e40fd3", "content-length": "179", "date": "Wed, 30 Apr 2025 06:44:02 GMT"} })
67322025-04-30T06:44:02.846ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1/component/rot/cmpa
67332025-04-30T06:44:02.848ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "90ace9ca-5161-4462-926d-8fc285c0100e", "content-length": "702", "date": "Wed, 30 Apr 2025 06:44:02 GMT"} })
67342025-04-30T06:44:02.848ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1/component/rot/cfpa
67352025-04-30T06:44:02.850ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "7810c470-5160-4ed9-9997-50de6b55f2bd", "content-length": "727", "date": "Wed, 30 Apr 2025 06:44:02 GMT"} })
67362025-04-30T06:44:02.850ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1/component/rot/cfpa
67372025-04-30T06:44:02.851ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "f5ef3848-0dd9-4cc6-845d-5e808fa11af7", "content-length": "729", "date": "Wed, 30 Apr 2025 06:44:02 GMT"} })
67382025-04-30T06:44:02.852ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1/component/rot/cfpa
67392025-04-30T06:44:02.853ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "33389d86-d466-4f14-9ab4-3de44b9218da", "content-length": "728", "date": "Wed, 30 Apr 2025 06:44:02 GMT"} })
67402025-04-30T06:44:02.853ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0
67412025-04-30T06:44:02.855ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "cce2e190-ec41-4ce2-9e4d-804a24e0fda6", "content-length": "723", "date": "Wed, 30 Apr 2025 06:44:02 GMT"} })
67422025-04-30T06:44:02.855ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0/component/sp/caboose?firmware_slot=0
67432025-04-30T06:44:02.858ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "67e7b835-4785-4f3d-8653-eff0f451e2a0", "content-length": "109", "date": "Wed, 30 Apr 2025 06:44:02 GMT"} })
67442025-04-30T06:44:02.858ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0/component/sp/caboose?firmware_slot=1
67452025-04-30T06:44:02.860ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "414f1fdf-e062-4ab3-8deb-7de53b7352f0", "content-length": "109", "date": "Wed, 30 Apr 2025 06:44:02 GMT"} })
67462025-04-30T06:44:02.861ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0/component/rot/caboose?firmware_slot=0
67472025-04-30T06:44:02.863ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "752826ed-b4cb-4193-897d-c7a41d2f65e6", "content-length": "169", "date": "Wed, 30 Apr 2025 06:44:02 GMT"} })
67482025-04-30T06:44:02.863ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0/component/rot/caboose?firmware_slot=1
67492025-04-30T06:44:02.865ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "ea2fd545-9c01-46a1-8a98-ec1399199580", "content-length": "169", "date": "Wed, 30 Apr 2025 06:44:02 GMT"} })
67502025-04-30T06:44:02.865ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0/component/stage0/caboose?firmware_slot=0
67512025-04-30T06:44:02.867ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "d61d092a-e7cf-4bd5-a887-c92ae738f479", "content-length": "178", "date": "Wed, 30 Apr 2025 06:44:02 GMT"} })
67522025-04-30T06:44:02.867ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0/component/stage0/caboose?firmware_slot=1
67532025-04-30T06:44:02.869ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "72ccafdb-767c-41d0-a4fd-a0a80a69b4fe", "content-length": "178", "date": "Wed, 30 Apr 2025 06:44:02 GMT"} })
67542025-04-30T06:44:02.869ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0/component/rot/cmpa
67552025-04-30T06:44:02.870ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "bb920f5e-3a9d-4d24-9451-2d18425937ea", "content-length": "702", "date": "Wed, 30 Apr 2025 06:44:02 GMT"} })
67562025-04-30T06:44:02.871ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0/component/rot/cfpa
67572025-04-30T06:44:02.872ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "b37669b8-a04f-4809-aba7-c946f1f25e0a", "content-length": "727", "date": "Wed, 30 Apr 2025 06:44:02 GMT"} })
67582025-04-30T06:44:02.872ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0/component/rot/cfpa
67592025-04-30T06:44:02.873ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "3cabef19-2ae8-4bf4-a51d-64b22ae467ba", "content-length": "729", "date": "Wed, 30 Apr 2025 06:44:02 GMT"} })
67602025-04-30T06:44:02.873ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0/component/rot/cfpa
67612025-04-30T06:44:02.874ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "da81ce59-12d1-4637-a344-11a457148ddb", "content-length": "728", "date": "Wed, 30 Apr 2025 06:44:02 GMT"} })
67622025-04-30T06:44:02.874ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1
67632025-04-30T06:44:02.876ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1", status: 200, headers: {"content-type": "application/json", "x-request-id": "5eaf0e75-a132-493d-afa6-5c7e315c26dc", "content-length": "723", "date": "Wed, 30 Apr 2025 06:44:02 GMT"} })
67642025-04-30T06:44:02.876ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1/component/sp/caboose?firmware_slot=0
67652025-04-30T06:44:02.879ZTRACtest_utilization (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory
67662025-04-30T06:44:02.879ZTRACtest_utilization (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
67672025-04-30T06:44:02.879ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
67682025-04-30T06:44:02.879ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
67692025-04-30T06:44:02.880ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
67702025-04-30T06:44:02.881ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "1e9b4280-2054-4279-b847-932828258576", "content-length": "109", "date": "Wed, 30 Apr 2025 06:44:02 GMT"} })
67712025-04-30T06:44:02.882ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1/component/sp/caboose?firmware_slot=1
67722025-04-30T06:44:02.884ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
67732025-04-30T06:44:02.889ZDEBGtest_utilization (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory result = Ok(())
67742025-04-30T06:44:02.890ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
67752025-04-30T06:44:02.890ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
67762025-04-30T06:44:02.890ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
67772025-04-30T06:44:02.892ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "5e545659-f0a5-426f-b4e0-7e8f67d61ee2", "content-length": "109", "date": "Wed, 30 Apr 2025 06:44:02 GMT"} })
67782025-04-30T06:44:02.893ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1/component/rot/caboose?firmware_slot=0
67792025-04-30T06:44:02.893ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
67802025-04-30T06:44:02.893ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
67812025-04-30T06:44:02.894ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
67822025-04-30T06:44:02.897ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "07b1ac7c-5c9f-41dc-a818-285e91d6813c", "content-length": "169", "date": "Wed, 30 Apr 2025 06:44:02 GMT"} })
67832025-04-30T06:44:02.897ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1/component/rot/caboose?firmware_slot=1
67842025-04-30T06:44:02.902ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "7d09c490-df53-48e1-90f9-f1bf5888f318", "content-length": "169", "date": "Wed, 30 Apr 2025 06:44:02 GMT"} })
67852025-04-30T06:44:02.902ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1/component/stage0/caboose?firmware_slot=0
67862025-04-30T06:44:02.906ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "ad7c7664-12a4-490b-a78f-cd44c231354f", "content-length": "178", "date": "Wed, 30 Apr 2025 06:44:02 GMT"} })
67872025-04-30T06:44:02.906ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1/component/stage0/caboose?firmware_slot=1
67882025-04-30T06:44:02.908ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "a73f0593-9191-47d5-abcc-bf2c1b863843", "content-length": "178", "date": "Wed, 30 Apr 2025 06:44:02 GMT"} })
67892025-04-30T06:44:02.908ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1/component/rot/cmpa
67902025-04-30T06:44:02.909ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "2be4393c-a903-4c0c-9f73-85b7cc227ed0", "content-length": "702", "date": "Wed, 30 Apr 2025 06:44:02 GMT"} })
67912025-04-30T06:44:02.910ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1/component/rot/cfpa
67922025-04-30T06:44:02.912ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "e2df5a20-ed87-49c0-81aa-7f7e73d81d11", "content-length": "727", "date": "Wed, 30 Apr 2025 06:44:02 GMT"} })
67932025-04-30T06:44:02.912ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1/component/rot/cfpa
67942025-04-30T06:44:02.914ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "7b5596e4-df5e-4cb3-9f36-8e05769757a2", "content-length": "729", "date": "Wed, 30 Apr 2025 06:44:02 GMT"} })
67952025-04-30T06:44:02.914ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1/component/rot/cfpa
67962025-04-30T06:44:02.915ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "7d9a0219-c68d-40cd-b026-f920d3875a1f", "content-length": "728", "date": "Wed, 30 Apr 2025 06:44:02 GMT"} })
67972025-04-30T06:44:02.916ZTRACtest_utilization (BackgroundTasks): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet
67982025-04-30T06:44:02.916ZTRACtest_utilization (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
67992025-04-30T06:44:02.916ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
68002025-04-30T06:44:02.916ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
68012025-04-30T06:44:02.917ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
68022025-04-30T06:44:02.920ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
68032025-04-30T06:44:02.930ZDEBGtest_utilization (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet result = Ok(())
68042025-04-30T06:44:02.930ZTRACtest_utilization (BackgroundTasks): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet
68052025-04-30T06:44:02.930ZTRACtest_utilization (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
68062025-04-30T06:44:02.930ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
68072025-04-30T06:44:02.930ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
68082025-04-30T06:44:02.930ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
68092025-04-30T06:44:02.944ZDEBGtest_utilization: attempt to fetch latest inventory collection result = Ok("found")
68102025-04-30T06:44:02.945ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
68112025-04-30T06:44:02.953ZDEBGtest_utilization (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet result = Ok(())
68122025-04-30T06:44:02.953ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
68132025-04-30T06:44:02.953ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
68142025-04-30T06:44:02.954ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
68152025-04-30T06:44:03.016ZDEBGtest_utilization (BackgroundTasks): begin collection from Sled Agent background_task = inventory_collection sled_agent_url = http://[::1]:39365
68162025-04-30T06:44:03.016ZDEBGtest_utilization (BackgroundTasks): client request SledAgent = http://[::1]:39365 background_task = inventory_collection body = None method = GET uri = http://[::1]:39365/inventory
68172025-04-30T06:44:03.017ZTRACtest_utilization (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory
68182025-04-30T06:44:03.017ZTRACtest_utilization (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
68192025-04-30T06:44:03.017ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
68202025-04-30T06:44:03.017ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
68212025-04-30T06:44:03.018ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
68222025-04-30T06:44:03.018ZINFOtest_utilization (dropshot): accepted connection local_addr = [::1]:39365 remote_addr = [::1]:46685 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
68232025-04-30T06:44:03.018ZDEBGtest_utilization (collection-task): sent timer-based collection request to the collection task collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
68242025-04-30T06:44:03.018ZDEBGtest_utilization (collection-task): collecting from producer collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
68252025-04-30T06:44:03.024ZTRACtest_utilization (dropshot): incoming request local_addr = [::1]:39365 method = GET remote_addr = [::1]:46685 req_id = 40fe4a5e-0d78-43aa-b0ee-f153c23395f2 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /inventory
68262025-04-30T06:44:03.026ZINFOtest_utilization (dropshot): request completed latency_us = 2604 local_addr = [::1]:39365 method = GET remote_addr = [::1]:46685 req_id = 40fe4a5e-0d78-43aa-b0ee-f153c23395f2 response_code = 200 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /inventory
68272025-04-30T06:44:03.027ZTRACtest_utilization (collection-task): sent collection request to producer collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
68282025-04-30T06:44:03.028ZDEBGtest_utilization (collection-task): collected results from producer collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 n_results = 1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
68292025-04-30T06:44:03.028ZTRACtest_utilization (collection-task): forwarded results to main collection loop collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
68302025-04-30T06:44:03.028ZTRACtest_utilization (collection-task): top of inner collection loop, waiting for next request collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
68312025-04-30T06:44:03.028ZDEBGtest_utilization (BackgroundTasks): client response SledAgent = http://[::1]:39365 background_task = inventory_collection result = Ok(Response { url: "http://[::1]:39365/inventory", status: 200, headers: {"content-type": "application/json", "x-request-id": "40fe4a5e-0d78-43aa-b0ee-f153c23395f2", "content-length": "3141", "date": "Wed, 30 Apr 2025 06:44:02 GMT"} })
68322025-04-30T06:44:03.030ZDEBGtest_utilization (BackgroundTasks): begin collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
68332025-04-30T06:44:03.030ZDEBGtest_utilization (BackgroundTasks): end collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
68342025-04-30T06:44:03.030ZDEBGtest_utilization (BackgroundTasks): finished collection background_task = inventory_collection
68352025-04-30T06:44:03.032ZTRACtest_utilization (BackgroundTasks): authorize begin action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory
68362025-04-30T06:44:03.032ZTRACtest_utilization (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
68372025-04-30T06:44:03.032ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
68382025-04-30T06:44:03.032ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
68392025-04-30T06:44:03.033ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
68402025-04-30T06:44:03.034ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
68412025-04-30T06:44:03.045ZDEBGtest_utilization (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory result = Ok(())
68422025-04-30T06:44:03.045ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
68432025-04-30T06:44:03.045ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
68442025-04-30T06:44:03.046ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
68452025-04-30T06:44:03.048ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
68462025-04-30T06:44:03.053ZDEBGtest_utilization (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory result = Ok(())
68472025-04-30T06:44:03.054ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
68482025-04-30T06:44:03.054ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
68492025-04-30T06:44:03.070ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
68502025-04-30T06:44:03.074ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
68512025-04-30T06:44:03.074ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
68522025-04-30T06:44:03.076ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
68532025-04-30T06:44:03.189ZDEBGtest_utilization: attempt to fetch latest inventory collection result = Ok("found")
68542025-04-30T06:44:03.225ZWARNtest_utilization (datastore): Non-retryable transaction failure err = DatabaseError(UniqueViolation, "duplicate key value violates unique constraint \\"inv_caboose_pkey\\"") transaction = inventory_insert_collection
68552025-04-30T06:44:03.225ZWARNtest_utilization (BackgroundTasks): inventory collection failed background_task = inventory_collection error = failed to collect inventory: saving inventory to database: Internal Error: unexpected database error: duplicate key value violates unique constraint "inv_caboose_pkey"
68562025-04-30T06:44:03.225ZDEBGtest_utilization (BackgroundTasks): activation complete background_task = inventory_collection elapsed = 576.844885ms iteration = 7 status = {"error":"failed to collect inventory: saving inventory to database: Internal Error: unexpected database error: duplicate key value violates unique constraint \\"inv_caboose_pkey\\""}
68572025-04-30T06:44:03.225ZDEBGtest_utilization (BackgroundTasks): activating background_task = inventory_collection iteration = 8 reason = Signaled
68582025-04-30T06:44:03.225ZTRACtest_utilization (BackgroundTasks): authorize begin action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory
68592025-04-30T06:44:03.226ZTRACtest_utilization (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
68602025-04-30T06:44:03.226ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
68612025-04-30T06:44:03.226ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
68622025-04-30T06:44:03.229ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
68632025-04-30T06:44:03.237ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
68642025-04-30T06:44:03.242ZDEBGtest_utilization (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory result = Ok(())
68652025-04-30T06:44:03.243ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
68662025-04-30T06:44:03.243ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
68672025-04-30T06:44:03.243ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
68682025-04-30T06:44:03.249ZTRACtest_utilization (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory
68692025-04-30T06:44:03.249ZTRACtest_utilization (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
68702025-04-30T06:44:03.249ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
68712025-04-30T06:44:03.249ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
68722025-04-30T06:44:03.250ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
68732025-04-30T06:44:03.255ZDEBGtest_utilization (BackgroundTasks): inventory_prune_one: nothing eligible for removal (too few) background_task = inventory_collection candidates = [(13d8b8d9-6c6b-47a6-9190-952544edb75b, 0)]
68742025-04-30T06:44:03.255ZTRACtest_utilization (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal
68752025-04-30T06:44:03.257ZTRACtest_utilization (dns): buffer buffer = Length: 67 (0x43) bytes\n0000: de d2 01 00 00 01 00 00 00 00 00 01 04 5f 6d 67 ............._mg\n0010: 73 04 5f 74 63 70 0d 63 6f 6e 74 72 6f 6c 2d 70 s._tcp.control-p\n0020: 6c 61 6e 65 05 6f 78 69 64 65 08 69 6e 74 65 72 lane.oxide.inter\n0030: 6e 61 6c 00 00 21 00 01 00 00 29 04 d0 00 00 00 nal..!....).....\n0040: 00 00 00 ... kind = dns peer_addr = [::1]:64716 req_id = ac950b0d-5d5a-4e9a-9064-2cf9303d8fe7
68762025-04-30T06:44:03.257ZDEBGtest_utilization (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 57042,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_mgs._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_mgs._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 4,\n 95,\n 109,\n 103,\n 115,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:64716 req_id = ac950b0d-5d5a-4e9a-9064-2cf9303d8fe7
68772025-04-30T06:44:03.258ZDEBGtest_utilization (store): query key key = _mgs._tcp
68782025-04-30T06:44:03.258ZDEBGtest_utilization (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
68792025-04-30T06:44:03.258ZDEBGtest_utilization (dns): dns response additional_records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal"), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] kind = dns peer_addr = [::1]:64716 query = LowerQuery { name: LowerName(Name("_mgs._tcp.control-plane.oxide.internal.")), original: Query { name: Name("_mgs._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN } } records = [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 48759, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal") })) }] req_id = ac950b0d-5d5a-4e9a-9064-2cf9303d8fe7
68802025-04-30T06:44:03.258ZDEBGtest_utilization (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgs._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 48759, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }], valid_until: Instant { tv_sec: 4308, tv_nsec: 57680636 } })
68812025-04-30T06:44:03.258ZTRACtest_utilization (DnsResolver): lookup_service_targets: looking up SRV target
68822025-04-30T06:44:03.259ZTRACtest_utilization (dns): buffer buffer = Length: 108 (0x6c) bytes\n0000: 61 bf 01 00 00 01 00 00 00 00 00 01 2d 64 65 6e a...........-den\n0010: 64 72 69 74 65 2d 62 36 64 36 35 33 34 31 2d 31 drite-b6d65341-1\n0020: 36 37 63 2d 34 31 64 66 2d 39 62 35 63 2d 34 31 67c-41df-9b5c-41\n0030: 63 64 65 64 39 39 63 32 32 39 04 68 6f 73 74 0d cded99c229.host.\n0040: 63 6f 6e 74 72 6f 6c 2d 70 6c 61 6e 65 05 6f 78 control-plane.ox\n0050: 69 64 65 08 69 6e 74 65 72 6e 61 6c 00 00 1c 00 ide.internal....\n0060: 01 00 00 29 04 d0 00 00 00 00 00 00 ...)........ kind = dns peer_addr = [::1]:54794 req_id = 31b6ece0-4ad7-401b-9058-789f60d55c80
68832025-04-30T06:44:03.260ZDEBGtest_utilization (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 25023,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n query_type: AAAA,\n query_class: IN,\n },\n },\n original: [\n 45,\n 100,\n 101,\n 110,\n 100,\n 114,\n 105,\n 116,\n 101,\n 45,\n 98,\n 54,\n 100,\n 54,\n 53,\n 51,\n 52,\n 49,\n 45,\n 49,\n 54,\n 55,\n 99,\n 45,\n 52,\n 49,\n 100,\n 102,\n 45,\n 57,\n 98,\n 53,\n 99,\n 45,\n 52,\n 49,\n 99,\n 100,\n 101,\n 100,\n 57,\n 57,\n 99,\n 50,\n 50,\n 57,\n 4,\n 104,\n 111,\n 115,\n 116,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 28,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:54794 req_id = 31b6ece0-4ad7-401b-9058-789f60d55c80
68842025-04-30T06:44:03.260ZDEBGtest_utilization (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
68852025-04-30T06:44:03.260ZDEBGtest_utilization (dns): dns response additional_records = [] kind = dns peer_addr = [::1]:54794 query = LowerQuery { name: LowerName(Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.")), original: Query { name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), query_type: AAAA, query_class: IN } } records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] req_id = 31b6ece0-4ad7-401b-9058-789f60d55c80
68862025-04-30T06:44:03.260ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
68872025-04-30T06:44:03.292ZDEBGtest_utilization (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory result = Ok(())
68882025-04-30T06:44:03.292ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
68892025-04-30T06:44:03.292ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
68902025-04-30T06:44:03.292ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
68912025-04-30T06:44:03.359ZTRACtest_utilization (DnsResolver): lookup_all_socket_v6 srv dns_name = _clickhouse-admin-keeper._tcp.control-plane.oxide.internal
68922025-04-30T06:44:03.360ZDEBGtest_utilization (results-sink): inserting 1 samples into database collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
68932025-04-30T06:44:03.360ZDEBGtest_utilization (results-sink-cluster): inserting 1 samples into database collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
68942025-04-30T06:44:03.360ZDEBGtest_utilization (clickhouse-client): unrolling 1 total samples collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 4d1485f0-110e-4161-9a33-c4610d68be5d
68952025-04-30T06:44:03.362ZTRACtest_utilization (clickhouse-client): inserting data collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 4d1485f0-110e-4161-9a33-c4610d68be5d n_columns = 4 n_rows = 2 sql = INSERT INTO oximeter.fields_string FORMAT Native
68962025-04-30T06:44:03.362ZTRACtest_utilization (dns): buffer buffer = Length: 87 (0x57) bytes\n0000: f3 8d 01 00 00 01 00 00 00 00 00 01 18 5f 63 6c ............._cl\n0010: 69 63 6b 68 6f 75 73 65 2d 61 64 6d 69 6e 2d 6b ickhouse-admin-k\n0020: 65 65 70 65 72 04 5f 74 63 70 0d 63 6f 6e 74 72 eeper._tcp.contr\n0030: 6f 6c 2d 70 6c 61 6e 65 05 6f 78 69 64 65 08 69 ol-plane.oxide.i\n0040: 6e 74 65 72 6e 61 6c 00 00 21 00 01 00 00 29 04 nternal..!....).\n0050: d0 00 00 00 00 00 00 ....... kind = dns peer_addr = [::1]:59853 req_id = 071bdf33-cf4e-40af-aca5-bde4eb6a7987
68972025-04-30T06:44:03.362ZDEBGtest_utilization (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 62349,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 24,\n 95,\n 99,\n 108,\n 105,\n 99,\n 107,\n 104,\n 111,\n 117,\n 115,\n 101,\n 45,\n 97,\n 100,\n 109,\n 105,\n 110,\n 45,\n 107,\n 101,\n 101,\n 112,\n 101,\n 114,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:59853 req_id = 071bdf33-cf4e-40af-aca5-bde4eb6a7987
68982025-04-30T06:44:03.362ZDEBGtest_utilization (store): query key key = _clickhouse-admin-keeper._tcp
68992025-04-30T06:44:03.363ZERROtest_utilization (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 62349, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 1, }, query: WireQuery { query: LowerQuery { name: LowerName( Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), ), original: Query { name: Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN, }, }, original: [ 24, 95, 99, 108, 105, 99, 107, 104, 111, 117, 115, 101, 45, 97, 100, 109, 105, 110, 45, 107, 101, 101, 112, 101, 114, 4, 95, 116, 99, 112, 13, 99, 111, 110, 116, 114, 111, 108, 45, 112, 108, 97, 110, 101, 5, 111, 120, 105, 100, 101, 8, 105, 110, 116, 101, 114, 110, 97, 108, 0, 0, 33, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: Some( Edns { rcode_high: 0, version: 0, dnssec_ok: false, max_payload: 1232, options: OPT { options: {}, }, }, ), } NXDOMAIN: no records found for name: "_clickhouse-admin-keeper._tcp.control-plane.oxide.internal." kind = dns peer_addr = [::1]:59853 req_id = 071bdf33-cf4e-40af-aca5-bde4eb6a7987
69002025-04-30T06:44:03.363ZDEBGtest_utilization (BackgroundTasks): begin collection background_task = inventory_collection
69012025-04-30T06:44:03.363ZDEBGtest_utilization (BackgroundTasks): begin collection from MGS background_task = inventory_collection mgs_url = http://[::1]:48759
69022025-04-30T06:44:03.363ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/ignition
69032025-04-30T06:44:03.365ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
69042025-04-30T06:44:03.365ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
69052025-04-30T06:44:03.366ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
69062025-04-30T06:44:03.370ZDEBGtest_utilization (clickhouse-client): inserted rows into table collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 4d1485f0-110e-4161-9a33-c4610d68be5d n_rows = 2 table_name = fields_string
69072025-04-30T06:44:03.370ZTRACtest_utilization (clickhouse-client): inserting data collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 4d1485f0-110e-4161-9a33-c4610d68be5d n_columns = 4 n_rows = 1 sql = INSERT INTO oximeter.measurements_i64 FORMAT Native
69082025-04-30T06:44:03.373ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/ignition", status: 200, headers: {"content-type": "application/json", "x-request-id": "634bc78f-5b9a-4ba6-9d69-21e7430fd0c7", "content-length": "835", "date": "Wed, 30 Apr 2025 06:44:03 GMT"} })
69092025-04-30T06:44:03.373ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0
69102025-04-30T06:44:03.375ZDEBGtest_utilization (clickhouse-client): inserted rows into table collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 4d1485f0-110e-4161-9a33-c4610d68be5d n_rows = 1 table_name = measurements_i64
69112025-04-30T06:44:03.375ZTRACtest_utilization (results-sink): successfully inserted samples collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
69122025-04-30T06:44:03.376ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "723e4753-ac09-4315-9cfb-48523db997dd", "content-length": "734", "date": "Wed, 30 Apr 2025 06:44:03 GMT"} })
69132025-04-30T06:44:03.377ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0/component/sp/caboose?firmware_slot=0
69142025-04-30T06:44:03.383ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "dd65bc23-482b-4065-842d-a02c2e68e7f4", "content-length": "111", "date": "Wed, 30 Apr 2025 06:44:03 GMT"} })
69152025-04-30T06:44:03.383ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0/component/sp/caboose?firmware_slot=1
69162025-04-30T06:44:03.387ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "8df0df7d-970c-41da-8e8b-9d22c1b55be4", "content-length": "111", "date": "Wed, 30 Apr 2025 06:44:03 GMT"} })
69172025-04-30T06:44:03.388ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0/component/rot/caboose?firmware_slot=0
69182025-04-30T06:44:03.391ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "de9dc026-48e9-40fa-9832-09eb66cb7a51", "content-length": "170", "date": "Wed, 30 Apr 2025 06:44:03 GMT"} })
69192025-04-30T06:44:03.391ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0/component/rot/caboose?firmware_slot=1
69202025-04-30T06:44:03.397ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "91a9815f-034b-47c9-a069-d5266165bfb4", "content-length": "170", "date": "Wed, 30 Apr 2025 06:44:03 GMT"} })
69212025-04-30T06:44:03.397ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0/component/stage0/caboose?firmware_slot=0
69222025-04-30T06:44:03.400ZDEBGtest_utilization: attempt to fetch latest inventory collection result = Ok("found")
69232025-04-30T06:44:03.400ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "45d4de1b-30ee-4c69-98d8-1e3d7cdbff74", "content-length": "179", "date": "Wed, 30 Apr 2025 06:44:03 GMT"} })
69242025-04-30T06:44:03.401ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0/component/stage0/caboose?firmware_slot=1
69252025-04-30T06:44:03.406ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "1438f80d-df3f-44c8-bdd5-03cc8f0aeb03", "content-length": "179", "date": "Wed, 30 Apr 2025 06:44:03 GMT"} })
69262025-04-30T06:44:03.407ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0/component/rot/cmpa
69272025-04-30T06:44:03.407ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "cc6e4d83-7195-4be2-b91f-a1fc9cfc273d", "content-length": "702", "date": "Wed, 30 Apr 2025 06:44:03 GMT"} })
69282025-04-30T06:44:03.408ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0/component/rot/cfpa
69292025-04-30T06:44:03.409ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "ccef7a3c-b911-4619-a792-4e0f6c80834a", "content-length": "727", "date": "Wed, 30 Apr 2025 06:44:03 GMT"} })
69302025-04-30T06:44:03.409ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0/component/rot/cfpa
69312025-04-30T06:44:03.410ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "4b03db84-a04e-4796-bf36-a51add5526ab", "content-length": "729", "date": "Wed, 30 Apr 2025 06:44:03 GMT"} })
69322025-04-30T06:44:03.410ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0/component/rot/cfpa
69332025-04-30T06:44:03.412ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "fcc7aecf-4a3c-4337-a354-adc6c4692584", "content-length": "728", "date": "Wed, 30 Apr 2025 06:44:03 GMT"} })
69342025-04-30T06:44:03.412ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1
69352025-04-30T06:44:03.414ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1", status: 200, headers: {"content-type": "application/json", "x-request-id": "29d62d33-655b-4372-84fe-6535e3918c5b", "content-length": "734", "date": "Wed, 30 Apr 2025 06:44:03 GMT"} })
69362025-04-30T06:44:03.415ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1/component/sp/caboose?firmware_slot=0
69372025-04-30T06:44:03.417ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "5f191d23-8ef3-42f0-9556-5ae484b48af4", "content-length": "111", "date": "Wed, 30 Apr 2025 06:44:03 GMT"} })
69382025-04-30T06:44:03.417ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1/component/sp/caboose?firmware_slot=1
69392025-04-30T06:44:03.420ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "c9bf0e24-25cb-4c16-a5ac-79f52f114770", "content-length": "111", "date": "Wed, 30 Apr 2025 06:44:03 GMT"} })
69402025-04-30T06:44:03.420ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1/component/rot/caboose?firmware_slot=0
69412025-04-30T06:44:03.424ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "6ff6c730-9bd9-4340-812c-8f49386d2c00", "content-length": "170", "date": "Wed, 30 Apr 2025 06:44:03 GMT"} })
69422025-04-30T06:44:03.424ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1/component/rot/caboose?firmware_slot=1
69432025-04-30T06:44:03.427ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "34702fac-5081-4b89-8110-73dcfb5104ed", "content-length": "170", "date": "Wed, 30 Apr 2025 06:44:03 GMT"} })
69442025-04-30T06:44:03.427ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1/component/stage0/caboose?firmware_slot=0
69452025-04-30T06:44:03.430ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "9d76349c-ebcb-497e-bdc6-849fccb63235", "content-length": "179", "date": "Wed, 30 Apr 2025 06:44:03 GMT"} })
69462025-04-30T06:44:03.430ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1/component/stage0/caboose?firmware_slot=1
69472025-04-30T06:44:03.433ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "8b68e856-d96e-4833-902a-6bdc303981fa", "content-length": "179", "date": "Wed, 30 Apr 2025 06:44:03 GMT"} })
69482025-04-30T06:44:03.433ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1/component/rot/cmpa
69492025-04-30T06:44:03.434ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "d18d57c6-9318-49fd-8c2c-d29e6d7dc094", "content-length": "702", "date": "Wed, 30 Apr 2025 06:44:03 GMT"} })
69502025-04-30T06:44:03.434ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1/component/rot/cfpa
69512025-04-30T06:44:03.436ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "5f7e99f2-9920-453b-8a33-72bb1718a95b", "content-length": "727", "date": "Wed, 30 Apr 2025 06:44:03 GMT"} })
69522025-04-30T06:44:03.436ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1/component/rot/cfpa
69532025-04-30T06:44:03.439ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "93449dde-5aee-4219-9cce-fd57815dab58", "content-length": "729", "date": "Wed, 30 Apr 2025 06:44:03 GMT"} })
69542025-04-30T06:44:03.439ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1/component/rot/cfpa
69552025-04-30T06:44:03.439ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "33d6c35e-f933-4b7c-99f2-9b971cdd8858", "content-length": "728", "date": "Wed, 30 Apr 2025 06:44:03 GMT"} })
69562025-04-30T06:44:03.439ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0
69572025-04-30T06:44:03.441ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "a5ea1d27-c8f0-4630-b910-380057545f9e", "content-length": "723", "date": "Wed, 30 Apr 2025 06:44:03 GMT"} })
69582025-04-30T06:44:03.442ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0/component/sp/caboose?firmware_slot=0
69592025-04-30T06:44:03.444ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "2fb4d46c-d700-4bc3-a91a-6a0656f00a4f", "content-length": "109", "date": "Wed, 30 Apr 2025 06:44:03 GMT"} })
69602025-04-30T06:44:03.444ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0/component/sp/caboose?firmware_slot=1
69612025-04-30T06:44:03.447ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "fc5ccbe6-0bf1-4d7f-b66e-a7bebb2f8c22", "content-length": "109", "date": "Wed, 30 Apr 2025 06:44:03 GMT"} })
69622025-04-30T06:44:03.447ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0/component/rot/caboose?firmware_slot=0
69632025-04-30T06:44:03.451ZTRACtest_utilization (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory
69642025-04-30T06:44:03.451ZTRACtest_utilization (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
69652025-04-30T06:44:03.451ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
69662025-04-30T06:44:03.451ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
69672025-04-30T06:44:03.451ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
69682025-04-30T06:44:03.455ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "18a5bc9d-a0fc-429a-bfea-e9e7a6b080fd", "content-length": "169", "date": "Wed, 30 Apr 2025 06:44:03 GMT"} })
69692025-04-30T06:44:03.455ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0/component/rot/caboose?firmware_slot=1
69702025-04-30T06:44:03.456ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
69712025-04-30T06:44:03.462ZDEBGtest_utilization (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory result = Ok(())
69722025-04-30T06:44:03.462ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
69732025-04-30T06:44:03.462ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
69742025-04-30T06:44:03.462ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
69752025-04-30T06:44:03.463ZWARNtest_utilization (results-sink-cluster): failed to insert some results into metric DB collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 error = Failed to check out connection to database: No backends found for this service file = oximeter/collector/src/results_sink.rs:92
69762025-04-30T06:44:03.465ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
69772025-04-30T06:44:03.465ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
69782025-04-30T06:44:03.465ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
69792025-04-30T06:44:03.466ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "a0f81bab-b49d-424c-a61a-6da334e7dac4", "content-length": "169", "date": "Wed, 30 Apr 2025 06:44:03 GMT"} })
69802025-04-30T06:44:03.466ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0/component/stage0/caboose?firmware_slot=0
69812025-04-30T06:44:03.469ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "090b0fbb-6b92-43a6-898a-5059395ed4fb", "content-length": "178", "date": "Wed, 30 Apr 2025 06:44:03 GMT"} })
69822025-04-30T06:44:03.469ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0/component/stage0/caboose?firmware_slot=1
69832025-04-30T06:44:03.471ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "a618ccd1-8291-44ed-af62-f5755d553a49", "content-length": "178", "date": "Wed, 30 Apr 2025 06:44:03 GMT"} })
69842025-04-30T06:44:03.471ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0/component/rot/cmpa
69852025-04-30T06:44:03.472ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "f1d92517-5124-495e-b07d-a06dbb1f161a", "content-length": "702", "date": "Wed, 30 Apr 2025 06:44:03 GMT"} })
69862025-04-30T06:44:03.473ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0/component/rot/cfpa
69872025-04-30T06:44:03.475ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "9d2b1a11-80f9-4f51-8591-0c9a0fa00ce9", "content-length": "727", "date": "Wed, 30 Apr 2025 06:44:03 GMT"} })
69882025-04-30T06:44:03.475ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0/component/rot/cfpa
69892025-04-30T06:44:03.480ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "e0786c02-687b-4178-9aac-c794477a6b62", "content-length": "729", "date": "Wed, 30 Apr 2025 06:44:03 GMT"} })
69902025-04-30T06:44:03.481ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0/component/rot/cfpa
69912025-04-30T06:44:03.483ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "7b096e74-2cbe-45c1-b044-cd4d88f49f95", "content-length": "728", "date": "Wed, 30 Apr 2025 06:44:03 GMT"} })
69922025-04-30T06:44:03.483ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1
69932025-04-30T06:44:03.483ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1", status: 200, headers: {"content-type": "application/json", "x-request-id": "3485d9ea-7d7b-4dc0-9950-82a2cdb0cf3e", "content-length": "723", "date": "Wed, 30 Apr 2025 06:44:03 GMT"} })
69942025-04-30T06:44:03.483ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1/component/sp/caboose?firmware_slot=0
69952025-04-30T06:44:03.485ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "d70ef794-0043-42e4-b3c0-289fcdf52d7f", "content-length": "109", "date": "Wed, 30 Apr 2025 06:44:03 GMT"} })
69962025-04-30T06:44:03.485ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1/component/sp/caboose?firmware_slot=1
69972025-04-30T06:44:03.488ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "f82f04f7-279e-4a2e-bb7f-6f0ac8414ed0", "content-length": "109", "date": "Wed, 30 Apr 2025 06:44:03 GMT"} })
69982025-04-30T06:44:03.488ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1/component/rot/caboose?firmware_slot=0
69992025-04-30T06:44:03.491ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "490493a4-a31d-40d9-915e-57f85c8cedca", "content-length": "169", "date": "Wed, 30 Apr 2025 06:44:03 GMT"} })
70002025-04-30T06:44:03.491ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1/component/rot/caboose?firmware_slot=1
70012025-04-30T06:44:03.497ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "1589dc9a-4e31-4c1e-a141-164b7f868052", "content-length": "169", "date": "Wed, 30 Apr 2025 06:44:03 GMT"} })
70022025-04-30T06:44:03.497ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1/component/stage0/caboose?firmware_slot=0
70032025-04-30T06:44:03.500ZDEBGtest_utilization: attempt to fetch latest inventory collection result = Ok("found")
70042025-04-30T06:44:03.513ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "92eac011-ec2c-4a25-858f-360358863725", "content-length": "178", "date": "Wed, 30 Apr 2025 06:44:03 GMT"} })
70052025-04-30T06:44:03.513ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1/component/stage0/caboose?firmware_slot=1
70062025-04-30T06:44:03.515ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "8f6a16f3-2723-44f6-ba4b-5b06a0abf229", "content-length": "178", "date": "Wed, 30 Apr 2025 06:44:03 GMT"} })
70072025-04-30T06:44:03.515ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1/component/rot/cmpa
70082025-04-30T06:44:03.516ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "ec53b43d-1231-4a4c-b9e5-9db1b96e2c1a", "content-length": "702", "date": "Wed, 30 Apr 2025 06:44:03 GMT"} })
70092025-04-30T06:44:03.516ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1/component/rot/cfpa
70102025-04-30T06:44:03.517ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "a723f0cb-0431-4ec4-b43c-253270f598b7", "content-length": "727", "date": "Wed, 30 Apr 2025 06:44:03 GMT"} })
70112025-04-30T06:44:03.517ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1/component/rot/cfpa
70122025-04-30T06:44:03.518ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "d5020054-ed0c-4f97-90fe-ee7b09fc8cd4", "content-length": "729", "date": "Wed, 30 Apr 2025 06:44:03 GMT"} })
70132025-04-30T06:44:03.518ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1/component/rot/cfpa
70142025-04-30T06:44:03.519ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "d262de4a-5bc2-482f-8361-19426b050f9a", "content-length": "728", "date": "Wed, 30 Apr 2025 06:44:03 GMT"} })
70152025-04-30T06:44:03.519ZTRACtest_utilization (BackgroundTasks): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet
70162025-04-30T06:44:03.520ZTRACtest_utilization (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
70172025-04-30T06:44:03.520ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
70182025-04-30T06:44:03.520ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
70192025-04-30T06:44:03.520ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
70202025-04-30T06:44:03.533ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
70212025-04-30T06:44:03.538ZDEBGtest_utilization (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet result = Ok(())
70222025-04-30T06:44:03.538ZTRACtest_utilization (BackgroundTasks): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet
70232025-04-30T06:44:03.538ZTRACtest_utilization (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
70242025-04-30T06:44:03.538ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
70252025-04-30T06:44:03.538ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
70262025-04-30T06:44:03.539ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
70272025-04-30T06:44:03.541ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
70282025-04-30T06:44:03.548ZDEBGtest_utilization (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet result = Ok(())
70292025-04-30T06:44:03.548ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
70302025-04-30T06:44:03.548ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
70312025-04-30T06:44:03.548ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
70322025-04-30T06:44:03.550ZTRACtest_utilization (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory
70332025-04-30T06:44:03.550ZTRACtest_utilization (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
70342025-04-30T06:44:03.550ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
70352025-04-30T06:44:03.550ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
70362025-04-30T06:44:03.551ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
70372025-04-30T06:44:03.594ZDEBGtest_utilization (BackgroundTasks): begin collection from Sled Agent background_task = inventory_collection sled_agent_url = http://[::1]:39365
70382025-04-30T06:44:03.594ZDEBGtest_utilization (BackgroundTasks): client request SledAgent = http://[::1]:39365 background_task = inventory_collection body = None method = GET uri = http://[::1]:39365/inventory
70392025-04-30T06:44:03.594ZINFOtest_utilization (dropshot): accepted connection local_addr = [::1]:39365 remote_addr = [::1]:59705 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
70402025-04-30T06:44:03.597ZTRACtest_utilization (dropshot): incoming request local_addr = [::1]:39365 method = GET remote_addr = [::1]:59705 req_id = b8086dc0-a75d-4932-a079-6de5953cb4a2 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /inventory
70412025-04-30T06:44:03.598ZINFOtest_utilization (dropshot): request completed latency_us = 607 local_addr = [::1]:39365 method = GET remote_addr = [::1]:59705 req_id = b8086dc0-a75d-4932-a079-6de5953cb4a2 response_code = 200 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /inventory
70422025-04-30T06:44:03.598ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
70432025-04-30T06:44:03.607ZDEBGtest_utilization (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory result = Ok(())
70442025-04-30T06:44:03.607ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
70452025-04-30T06:44:03.608ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
70462025-04-30T06:44:03.608ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
70472025-04-30T06:44:03.609ZDEBGtest_utilization (BackgroundTasks): client response SledAgent = http://[::1]:39365 background_task = inventory_collection result = Ok(Response { url: "http://[::1]:39365/inventory", status: 200, headers: {"content-type": "application/json", "x-request-id": "b8086dc0-a75d-4932-a079-6de5953cb4a2", "content-length": "3141", "date": "Wed, 30 Apr 2025 06:44:03 GMT"} })
70482025-04-30T06:44:03.614ZDEBGtest_utilization (BackgroundTasks): begin collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
70492025-04-30T06:44:03.614ZDEBGtest_utilization (BackgroundTasks): end collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
70502025-04-30T06:44:03.614ZDEBGtest_utilization (BackgroundTasks): finished collection background_task = inventory_collection
70512025-04-30T06:44:03.615ZTRACtest_utilization (BackgroundTasks): authorize begin action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory
70522025-04-30T06:44:03.615ZTRACtest_utilization (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
70532025-04-30T06:44:03.615ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
70542025-04-30T06:44:03.615ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
70552025-04-30T06:44:03.616ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
70562025-04-30T06:44:03.627ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
70572025-04-30T06:44:03.627ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
70582025-04-30T06:44:03.628ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
70592025-04-30T06:44:03.632ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
70602025-04-30T06:44:03.638ZDEBGtest_utilization (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory result = Ok(())
70612025-04-30T06:44:03.638ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
70622025-04-30T06:44:03.638ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
70632025-04-30T06:44:03.639ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
70642025-04-30T06:44:03.707ZWARNtest_utilization (datastore): Non-retryable transaction failure err = DatabaseError(UniqueViolation, "duplicate key value violates unique constraint \\"inv_caboose_pkey\\"") transaction = inventory_insert_collection
70652025-04-30T06:44:03.707ZWARNtest_utilization (BackgroundTasks): inventory collection failed background_task = inventory_collection error = failed to collect inventory: saving inventory to database: Internal Error: unexpected database error: duplicate key value violates unique constraint "inv_caboose_pkey"
70662025-04-30T06:44:03.707ZDEBGtest_utilization (BackgroundTasks): activation complete background_task = inventory_collection elapsed = 482.169362ms iteration = 8 status = {"error":"failed to collect inventory: saving inventory to database: Internal Error: unexpected database error: duplicate key value violates unique constraint \\"inv_caboose_pkey\\""}
70672025-04-30T06:44:03.707ZDEBGtest_utilization (BackgroundTasks): activating background_task = inventory_collection iteration = 9 reason = Signaled
70682025-04-30T06:44:03.708ZTRACtest_utilization (BackgroundTasks): authorize begin action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory
70692025-04-30T06:44:03.708ZTRACtest_utilization (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
70702025-04-30T06:44:03.708ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
70712025-04-30T06:44:03.708ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
70722025-04-30T06:44:03.709ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
70732025-04-30T06:44:03.712ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
70742025-04-30T06:44:03.719ZDEBGtest_utilization (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory result = Ok(())
70752025-04-30T06:44:03.719ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
70762025-04-30T06:44:03.719ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
70772025-04-30T06:44:03.721ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
70782025-04-30T06:44:03.730ZDEBGtest_utilization: attempt to fetch latest inventory collection result = Ok("found")
70792025-04-30T06:44:03.769ZDEBGtest_utilization (BackgroundTasks): inventory_prune_one: nothing eligible for removal (too few) background_task = inventory_collection candidates = [(13d8b8d9-6c6b-47a6-9190-952544edb75b, 0)]
70802025-04-30T06:44:03.769ZTRACtest_utilization (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal
70812025-04-30T06:44:03.770ZTRACtest_utilization (dns): buffer buffer = Length: 67 (0x43) bytes\n0000: 10 c4 01 00 00 01 00 00 00 00 00 01 04 5f 6d 67 ............._mg\n0010: 73 04 5f 74 63 70 0d 63 6f 6e 74 72 6f 6c 2d 70 s._tcp.control-p\n0020: 6c 61 6e 65 05 6f 78 69 64 65 08 69 6e 74 65 72 lane.oxide.inter\n0030: 6e 61 6c 00 00 21 00 01 00 00 29 04 d0 00 00 00 nal..!....).....\n0040: 00 00 00 ... kind = dns peer_addr = [::1]:52621 req_id = 875bc082-600f-4c5d-b334-b15cff852c7b
70822025-04-30T06:44:03.770ZDEBGtest_utilization (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 4292,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_mgs._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_mgs._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 4,\n 95,\n 109,\n 103,\n 115,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:52621 req_id = 875bc082-600f-4c5d-b334-b15cff852c7b
70832025-04-30T06:44:03.770ZDEBGtest_utilization (store): query key key = _mgs._tcp
70842025-04-30T06:44:03.771ZDEBGtest_utilization (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
70852025-04-30T06:44:03.772ZDEBGtest_utilization (dns): dns response additional_records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal"), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] kind = dns peer_addr = [::1]:52621 query = LowerQuery { name: LowerName(Name("_mgs._tcp.control-plane.oxide.internal.")), original: Query { name: Name("_mgs._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN } } records = [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 48759, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal") })) }] req_id = 875bc082-600f-4c5d-b334-b15cff852c7b
70862025-04-30T06:44:03.772ZDEBGtest_utilization (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgs._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 48759, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }], valid_until: Instant { tv_sec: 4308, tv_nsec: 571078217 } })
70872025-04-30T06:44:03.772ZTRACtest_utilization (DnsResolver): lookup_service_targets: looking up SRV target
70882025-04-30T06:44:03.773ZTRACtest_utilization (dns): buffer buffer = Length: 108 (0x6c) bytes\n0000: 4d b4 01 00 00 01 00 00 00 00 00 01 2d 64 65 6e M...........-den\n0010: 64 72 69 74 65 2d 62 36 64 36 35 33 34 31 2d 31 drite-b6d65341-1\n0020: 36 37 63 2d 34 31 64 66 2d 39 62 35 63 2d 34 31 67c-41df-9b5c-41\n0030: 63 64 65 64 39 39 63 32 32 39 04 68 6f 73 74 0d cded99c229.host.\n0040: 63 6f 6e 74 72 6f 6c 2d 70 6c 61 6e 65 05 6f 78 control-plane.ox\n0050: 69 64 65 08 69 6e 74 65 72 6e 61 6c 00 00 1c 00 ide.internal....\n0060: 01 00 00 29 04 d0 00 00 00 00 00 00 ...)........ kind = dns peer_addr = [::1]:55059 req_id = da141fae-26b7-4a23-bc68-7c653bdcc87d
70892025-04-30T06:44:03.773ZDEBGtest_utilization (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 19892,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n query_type: AAAA,\n query_class: IN,\n },\n },\n original: [\n 45,\n 100,\n 101,\n 110,\n 100,\n 114,\n 105,\n 116,\n 101,\n 45,\n 98,\n 54,\n 100,\n 54,\n 53,\n 51,\n 52,\n 49,\n 45,\n 49,\n 54,\n 55,\n 99,\n 45,\n 52,\n 49,\n 100,\n 102,\n 45,\n 57,\n 98,\n 53,\n 99,\n 45,\n 52,\n 49,\n 99,\n 100,\n 101,\n 100,\n 57,\n 57,\n 99,\n 50,\n 50,\n 57,\n 4,\n 104,\n 111,\n 115,\n 116,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 28,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:55059 req_id = da141fae-26b7-4a23-bc68-7c653bdcc87d
70902025-04-30T06:44:03.773ZDEBGtest_utilization (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
70912025-04-30T06:44:03.773ZDEBGtest_utilization (dns): dns response additional_records = [] kind = dns peer_addr = [::1]:55059 query = LowerQuery { name: LowerName(Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.")), original: Query { name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), query_type: AAAA, query_class: IN } } records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] req_id = da141fae-26b7-4a23-bc68-7c653bdcc87d
70922025-04-30T06:44:03.844ZTRACtest_utilization (DnsResolver): lookup_all_socket_v6 srv dns_name = _clickhouse-admin-keeper._tcp.control-plane.oxide.internal
70932025-04-30T06:44:03.845ZTRACtest_utilization (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory
70942025-04-30T06:44:03.845ZTRACtest_utilization (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
70952025-04-30T06:44:03.845ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
70962025-04-30T06:44:03.845ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
70972025-04-30T06:44:03.846ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
70982025-04-30T06:44:03.849ZTRACtest_utilization (dns): buffer buffer = Length: 87 (0x57) bytes\n0000: 1e 69 01 00 00 01 00 00 00 00 00 01 18 5f 63 6c .i..........._cl\n0010: 69 63 6b 68 6f 75 73 65 2d 61 64 6d 69 6e 2d 6b ickhouse-admin-k\n0020: 65 65 70 65 72 04 5f 74 63 70 0d 63 6f 6e 74 72 eeper._tcp.contr\n0030: 6f 6c 2d 70 6c 61 6e 65 05 6f 78 69 64 65 08 69 ol-plane.oxide.i\n0040: 6e 74 65 72 6e 61 6c 00 00 21 00 01 00 00 29 04 nternal..!....).\n0050: d0 00 00 00 00 00 00 ....... kind = dns peer_addr = [::1]:63823 req_id = ce72e2fa-2160-41c5-be1d-edeacbdcd08a
70992025-04-30T06:44:03.849ZDEBGtest_utilization (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 7785,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 24,\n 95,\n 99,\n 108,\n 105,\n 99,\n 107,\n 104,\n 111,\n 117,\n 115,\n 101,\n 45,\n 97,\n 100,\n 109,\n 105,\n 110,\n 45,\n 107,\n 101,\n 101,\n 112,\n 101,\n 114,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:63823 req_id = ce72e2fa-2160-41c5-be1d-edeacbdcd08a
71002025-04-30T06:44:03.849ZDEBGtest_utilization (store): query key key = _clickhouse-admin-keeper._tcp
71012025-04-30T06:44:03.849ZERROtest_utilization (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 7785, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 1, }, query: WireQuery { query: LowerQuery { name: LowerName( Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), ), original: Query { name: Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN, }, }, original: [ 24, 95, 99, 108, 105, 99, 107, 104, 111, 117, 115, 101, 45, 97, 100, 109, 105, 110, 45, 107, 101, 101, 112, 101, 114, 4, 95, 116, 99, 112, 13, 99, 111, 110, 116, 114, 111, 108, 45, 112, 108, 97, 110, 101, 5, 111, 120, 105, 100, 101, 8, 105, 110, 116, 101, 114, 110, 97, 108, 0, 0, 33, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: Some( Edns { rcode_high: 0, version: 0, dnssec_ok: false, max_payload: 1232, options: OPT { options: {}, }, }, ), } NXDOMAIN: no records found for name: "_clickhouse-admin-keeper._tcp.control-plane.oxide.internal." kind = dns peer_addr = [::1]:63823 req_id = ce72e2fa-2160-41c5-be1d-edeacbdcd08a
71022025-04-30T06:44:03.850ZDEBGtest_utilization (BackgroundTasks): begin collection background_task = inventory_collection
71032025-04-30T06:44:03.850ZDEBGtest_utilization (BackgroundTasks): begin collection from MGS background_task = inventory_collection mgs_url = http://[::1]:48759
71042025-04-30T06:44:03.850ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/ignition
71052025-04-30T06:44:03.853ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
71062025-04-30T06:44:03.860ZDEBGtest_utilization (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory result = Ok(())
71072025-04-30T06:44:03.860ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
71082025-04-30T06:44:03.860ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
71092025-04-30T06:44:03.861ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
71102025-04-30T06:44:03.869ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
71112025-04-30T06:44:03.869ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
71122025-04-30T06:44:03.869ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
71132025-04-30T06:44:03.869ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/ignition", status: 200, headers: {"content-type": "application/json", "x-request-id": "ea46998c-5434-4e79-a026-8b3fd5af29d1", "content-length": "835", "date": "Wed, 30 Apr 2025 06:44:03 GMT"} })
71142025-04-30T06:44:03.870ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0
71152025-04-30T06:44:03.872ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "036f59ec-f991-47a5-a045-e263e5e484dc", "content-length": "734", "date": "Wed, 30 Apr 2025 06:44:03 GMT"} })
71162025-04-30T06:44:03.872ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0/component/sp/caboose?firmware_slot=0
71172025-04-30T06:44:03.876ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "d33a63ad-af8d-4974-a739-5d3f0c91f12a", "content-length": "111", "date": "Wed, 30 Apr 2025 06:44:03 GMT"} })
71182025-04-30T06:44:03.877ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0/component/sp/caboose?firmware_slot=1
71192025-04-30T06:44:03.882ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "7e6f4ee3-6c39-4aa8-b427-447722f9783f", "content-length": "111", "date": "Wed, 30 Apr 2025 06:44:03 GMT"} })
71202025-04-30T06:44:03.883ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0/component/rot/caboose?firmware_slot=0
71212025-04-30T06:44:03.885ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "844a6c4f-f6be-489b-a7e1-1c6f8f4a65a2", "content-length": "170", "date": "Wed, 30 Apr 2025 06:44:03 GMT"} })
71222025-04-30T06:44:03.885ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0/component/rot/caboose?firmware_slot=1
71232025-04-30T06:44:03.900ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "0137739c-823f-41eb-a9ea-672024534f87", "content-length": "170", "date": "Wed, 30 Apr 2025 06:44:03 GMT"} })
71242025-04-30T06:44:03.900ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0/component/stage0/caboose?firmware_slot=0
71252025-04-30T06:44:03.907ZDEBGtest_utilization: attempt to fetch latest inventory collection result = Ok("found")
71262025-04-30T06:44:03.907ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "f5038dc5-c023-4ecd-adc7-1ea9a4d81809", "content-length": "179", "date": "Wed, 30 Apr 2025 06:44:03 GMT"} })
71272025-04-30T06:44:03.907ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0/component/stage0/caboose?firmware_slot=1
71282025-04-30T06:44:03.911ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "b8bf65f3-3dfb-4b3a-874e-8abc9c0b10d6", "content-length": "179", "date": "Wed, 30 Apr 2025 06:44:03 GMT"} })
71292025-04-30T06:44:03.911ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0/component/rot/cmpa
71302025-04-30T06:44:03.913ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "440108a6-ad90-438d-8dce-d06a6dc338af", "content-length": "702", "date": "Wed, 30 Apr 2025 06:44:03 GMT"} })
71312025-04-30T06:44:03.913ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0/component/rot/cfpa
71322025-04-30T06:44:03.914ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "d27447fc-0557-420f-8cdd-ffd9da9b539d", "content-length": "727", "date": "Wed, 30 Apr 2025 06:44:03 GMT"} })
71332025-04-30T06:44:03.914ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0/component/rot/cfpa
71342025-04-30T06:44:03.916ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "88d58b00-a202-46fa-a06e-f489646fbde5", "content-length": "729", "date": "Wed, 30 Apr 2025 06:44:03 GMT"} })
71352025-04-30T06:44:03.917ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0/component/rot/cfpa
71362025-04-30T06:44:03.918ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "8caab678-8ffb-4c42-a85b-2d8b74819a95", "content-length": "728", "date": "Wed, 30 Apr 2025 06:44:03 GMT"} })
71372025-04-30T06:44:03.918ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1
71382025-04-30T06:44:03.921ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1", status: 200, headers: {"content-type": "application/json", "x-request-id": "24579ead-b7a4-4f4e-ad6e-f9d0c5dd37d7", "content-length": "734", "date": "Wed, 30 Apr 2025 06:44:03 GMT"} })
71392025-04-30T06:44:03.921ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1/component/sp/caboose?firmware_slot=0
71402025-04-30T06:44:03.924ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "cd1f2eca-71cb-473c-b0c1-2d672c7c283a", "content-length": "111", "date": "Wed, 30 Apr 2025 06:44:03 GMT"} })
71412025-04-30T06:44:03.925ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1/component/sp/caboose?firmware_slot=1
71422025-04-30T06:44:03.927ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "481bb1cf-5ff7-4cf6-aeee-36478e7093aa", "content-length": "111", "date": "Wed, 30 Apr 2025 06:44:03 GMT"} })
71432025-04-30T06:44:03.927ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1/component/rot/caboose?firmware_slot=0
71442025-04-30T06:44:03.929ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "0efca0e8-6698-4626-b7d5-322f5cfd7e22", "content-length": "170", "date": "Wed, 30 Apr 2025 06:44:03 GMT"} })
71452025-04-30T06:44:03.929ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1/component/rot/caboose?firmware_slot=1
71462025-04-30T06:44:03.931ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "698f8437-68b6-4c0c-b253-97df79fd4b40", "content-length": "170", "date": "Wed, 30 Apr 2025 06:44:03 GMT"} })
71472025-04-30T06:44:03.931ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1/component/stage0/caboose?firmware_slot=0
71482025-04-30T06:44:03.933ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "902026b2-95fa-4791-a0e2-e17b1b5a11fe", "content-length": "179", "date": "Wed, 30 Apr 2025 06:44:03 GMT"} })
71492025-04-30T06:44:03.933ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1/component/stage0/caboose?firmware_slot=1
71502025-04-30T06:44:03.935ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "762ccc2e-9fea-4075-88a7-aade01faa990", "content-length": "179", "date": "Wed, 30 Apr 2025 06:44:03 GMT"} })
71512025-04-30T06:44:03.936ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1/component/rot/cmpa
71522025-04-30T06:44:03.939ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "a17f3414-9eb9-446b-bfdc-ed0319786b67", "content-length": "702", "date": "Wed, 30 Apr 2025 06:44:03 GMT"} })
71532025-04-30T06:44:03.939ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1/component/rot/cfpa
71542025-04-30T06:44:03.941ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "4226b365-e24f-4454-840f-7d61717d2444", "content-length": "727", "date": "Wed, 30 Apr 2025 06:44:03 GMT"} })
71552025-04-30T06:44:03.941ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1/component/rot/cfpa
71562025-04-30T06:44:03.943ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "077a34f2-ee56-491f-8782-12ffc6790e0d", "content-length": "729", "date": "Wed, 30 Apr 2025 06:44:03 GMT"} })
71572025-04-30T06:44:03.943ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1/component/rot/cfpa
71582025-04-30T06:44:03.944ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "e0e42a4e-fee1-4d9c-b3d2-4765908bcff9", "content-length": "728", "date": "Wed, 30 Apr 2025 06:44:03 GMT"} })
71592025-04-30T06:44:03.944ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0
71602025-04-30T06:44:03.947ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "4ba64d19-f7d1-4010-b793-513b9729bebe", "content-length": "723", "date": "Wed, 30 Apr 2025 06:44:03 GMT"} })
71612025-04-30T06:44:03.947ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0/component/sp/caboose?firmware_slot=0
71622025-04-30T06:44:03.950ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "6c132751-02d1-41a1-bfee-8ff5d1430a47", "content-length": "109", "date": "Wed, 30 Apr 2025 06:44:03 GMT"} })
71632025-04-30T06:44:03.950ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0/component/sp/caboose?firmware_slot=1
71642025-04-30T06:44:03.953ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "4c169901-1121-423a-b02d-a4740ca4fa81", "content-length": "109", "date": "Wed, 30 Apr 2025 06:44:03 GMT"} })
71652025-04-30T06:44:03.953ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0/component/rot/caboose?firmware_slot=0
71662025-04-30T06:44:03.956ZINFOtest_utilization (oximeter-agent): refreshing list of producers from Nexus collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 file = oximeter/collector/src/agent.rs:562
71672025-04-30T06:44:03.957ZDEBGtest_utilization (oximeter): client request body = None method = GET uri = http://127.0.0.1:32956/metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?sort_by=id_ascending
71682025-04-30T06:44:03.958ZTRACtest_utilization (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory
71692025-04-30T06:44:03.958ZTRACtest_utilization (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
71702025-04-30T06:44:03.958ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
71712025-04-30T06:44:03.958ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
71722025-04-30T06:44:03.959ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
71732025-04-30T06:44:03.959ZTRACtest_utilization (dropshot_internal): incoming request local_addr = 127.0.0.1:32956 method = GET remote_addr = 127.0.0.1:55797 req_id = b6f30893-62ca-4065-b2ca-92d6abb52344 uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?sort_by=id_ascending
71742025-04-30T06:44:03.959ZTRACtest_utilization (dropshot_internal): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = 127.0.0.1:32956 method = GET remote_addr = 127.0.0.1:55797 req_id = b6f30893-62ca-4065-b2ca-92d6abb52344 resource = Database uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?sort_by=id_ascending
71752025-04-30T06:44:03.959ZDEBGtest_utilization (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = 127.0.0.1:32956 method = GET remote_addr = 127.0.0.1:55797 req_id = b6f30893-62ca-4065-b2ca-92d6abb52344 roles = RoleSet { roles: {} } uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?sort_by=id_ascending
71762025-04-30T06:44:03.960ZDEBGtest_utilization (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = 127.0.0.1:32956 method = GET remote_addr = 127.0.0.1:55797 req_id = b6f30893-62ca-4065-b2ca-92d6abb52344 resource = Database result = Ok(()) uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?sort_by=id_ascending
71772025-04-30T06:44:03.965ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
71782025-04-30T06:44:03.978ZDEBGtest_utilization (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory result = Ok(())
71792025-04-30T06:44:03.978ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
71802025-04-30T06:44:03.978ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
71812025-04-30T06:44:03.979ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
71822025-04-30T06:44:03.979ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "ce9cc15f-3d02-4268-854e-f5219c4ad993", "content-length": "169", "date": "Wed, 30 Apr 2025 06:44:03 GMT"} })
71832025-04-30T06:44:03.979ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0/component/rot/caboose?firmware_slot=1
71842025-04-30T06:44:03.983ZINFOtest_utilization (dropshot_internal): request completed latency_us = 23553 local_addr = 127.0.0.1:32956 method = GET remote_addr = 127.0.0.1:55797 req_id = b6f30893-62ca-4065-b2ca-92d6abb52344 response_code = 200 uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?sort_by=id_ascending
71852025-04-30T06:44:03.983ZDEBGtest_utilization (oximeter): client response result = Ok(Response { url: "http://127.0.0.1:32956/metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?sort_by=id_ascending", status: 200, headers: {"content-type": "application/json", "x-request-id": "b6f30893-62ca-4065-b2ca-92d6abb52344", "content-length": "405", "date": "Wed, 30 Apr 2025 06:44:03 GMT"} })
71862025-04-30T06:44:03.983ZDEBGtest_utilization (oximeter): client request body = None method = GET uri = http://127.0.0.1:32956/metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJhYzU1ZDhkOS1hMjk4LTQ1MzQtODk4Yi1iZWVlMWM5ZTRlNTIifX0%3D
71872025-04-30T06:44:03.985ZTRACtest_utilization (dropshot_internal): incoming request local_addr = 127.0.0.1:32956 method = GET remote_addr = 127.0.0.1:55797 req_id = 70358448-19f1-4b78-bb49-dcbaa79bc38d uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJhYzU1ZDhkOS1hMjk4LTQ1MzQtODk4Yi1iZWVlMWM5ZTRlNTIifX0%3D
71882025-04-30T06:44:03.985ZTRACtest_utilization (dropshot_internal): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = 127.0.0.1:32956 method = GET remote_addr = 127.0.0.1:55797 req_id = 70358448-19f1-4b78-bb49-dcbaa79bc38d resource = Database uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJhYzU1ZDhkOS1hMjk4LTQ1MzQtODk4Yi1iZWVlMWM5ZTRlNTIifX0%3D
71892025-04-30T06:44:03.986ZDEBGtest_utilization (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = 127.0.0.1:32956 method = GET remote_addr = 127.0.0.1:55797 req_id = 70358448-19f1-4b78-bb49-dcbaa79bc38d roles = RoleSet { roles: {} } uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJhYzU1ZDhkOS1hMjk4LTQ1MzQtODk4Yi1iZWVlMWM5ZTRlNTIifX0%3D
71902025-04-30T06:44:03.986ZDEBGtest_utilization (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = 127.0.0.1:32956 method = GET remote_addr = 127.0.0.1:55797 req_id = 70358448-19f1-4b78-bb49-dcbaa79bc38d resource = Database result = Ok(()) uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJhYzU1ZDhkOS1hMjk4LTQ1MzQtODk4Yi1iZWVlMWM5ZTRlNTIifX0%3D
71912025-04-30T06:44:03.987ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
71922025-04-30T06:44:03.987ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
71932025-04-30T06:44:03.987ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
71942025-04-30T06:44:03.990ZINFOtest_utilization (dropshot_internal): request completed latency_us = 5440 local_addr = 127.0.0.1:32956 method = GET remote_addr = 127.0.0.1:55797 req_id = 70358448-19f1-4b78-bb49-dcbaa79bc38d response_code = 200 uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJhYzU1ZDhkOS1hMjk4LTQ1MzQtODk4Yi1iZWVlMWM5ZTRlNTIifX0%3D
71952025-04-30T06:44:03.991ZDEBGtest_utilization (oximeter): client response result = Ok(Response { url: "http://127.0.0.1:32956/metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJhYzU1ZDhkOS1hMjk4LTQ1MzQtODk4Yi1iZWVlMWM5ZTRlNTIifX0%3D", status: 200, headers: {"content-type": "application/json", "x-request-id": "70358448-19f1-4b78-bb49-dcbaa79bc38d", "content-length": "29", "date": "Wed, 30 Apr 2025 06:44:03 GMT"} })
71962025-04-30T06:44:03.991ZTRACtest_utilization (oximeter-agent): ignoring request to update existing metric producer, since the endpoint information is the same as the existing collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
71972025-04-30T06:44:03.991ZTRACtest_utilization (oximeter-agent): ignoring request to update existing metric producer, since the endpoint information is the same as the existing collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = ac55d8d9-a298-4534-898b-beee1c9e4e52
71982025-04-30T06:44:03.991ZINFOtest_utilization (oximeter-agent): refreshed list of producers from Nexus collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 file = oximeter/collector/src/agent.rs:637 n_current_tasks = 2 n_pruned_tasks = 0
71992025-04-30T06:44:03.992ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "3a51fb63-8081-4663-9745-ee8aa6693af1", "content-length": "169", "date": "Wed, 30 Apr 2025 06:44:03 GMT"} })
72002025-04-30T06:44:03.993ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0/component/stage0/caboose?firmware_slot=0
72012025-04-30T06:44:03.995ZDEBGtest_utilization (collection-task): sent timer-based collection request to the collection task collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
72022025-04-30T06:44:03.995ZDEBGtest_utilization (collection-task): collecting from producer collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
72032025-04-30T06:44:03.998ZTRACtest_utilization (collection-task): sent collection request to producer collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
72042025-04-30T06:44:03.999ZDEBGtest_utilization (collection-task): collected results from producer collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 n_results = 1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
72052025-04-30T06:44:03.999ZTRACtest_utilization (collection-task): forwarded results to main collection loop collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
72062025-04-30T06:44:03.999ZTRACtest_utilization (collection-task): top of inner collection loop, waiting for next request collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
72072025-04-30T06:44:04.002ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "3a0bb239-9970-4ca7-ba97-0af9cb78db31", "content-length": "178", "date": "Wed, 30 Apr 2025 06:44:03 GMT"} })
72082025-04-30T06:44:04.002ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0/component/stage0/caboose?firmware_slot=1
72092025-04-30T06:44:04.005ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "7e690738-49d0-4a7c-a4c3-e6c80e7edb63", "content-length": "178", "date": "Wed, 30 Apr 2025 06:44:03 GMT"} })
72102025-04-30T06:44:04.005ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0/component/rot/cmpa
72112025-04-30T06:44:04.006ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "29f1d824-484e-4573-8c0c-2d58d065fd9e", "content-length": "702", "date": "Wed, 30 Apr 2025 06:44:03 GMT"} })
72122025-04-30T06:44:04.006ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0/component/rot/cfpa
72132025-04-30T06:44:04.008ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "efb212fe-abc0-411e-9dd3-13d52e981680", "content-length": "727", "date": "Wed, 30 Apr 2025 06:44:03 GMT"} })
72142025-04-30T06:44:04.008ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0/component/rot/cfpa
72152025-04-30T06:44:04.010ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "58145ba1-877b-4405-add8-909710969319", "content-length": "729", "date": "Wed, 30 Apr 2025 06:44:03 GMT"} })
72162025-04-30T06:44:04.011ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0/component/rot/cfpa
72172025-04-30T06:44:04.012ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "a535ccd7-0722-442a-a956-502bb13b51d1", "content-length": "728", "date": "Wed, 30 Apr 2025 06:44:03 GMT"} })
72182025-04-30T06:44:04.012ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1
72192025-04-30T06:44:04.014ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1", status: 200, headers: {"content-type": "application/json", "x-request-id": "73715cd1-3b2e-46c9-976f-cb3964228e26", "content-length": "723", "date": "Wed, 30 Apr 2025 06:44:03 GMT"} })
72202025-04-30T06:44:04.014ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1/component/sp/caboose?firmware_slot=0
72212025-04-30T06:44:04.022ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "3f292a9a-380c-4a0a-8fc6-3c2f65eca8e9", "content-length": "109", "date": "Wed, 30 Apr 2025 06:44:03 GMT"} })
72222025-04-30T06:44:04.023ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1/component/sp/caboose?firmware_slot=1
72232025-04-30T06:44:04.024ZDEBGtest_utilization: attempt to fetch latest inventory collection result = Ok("found")
72242025-04-30T06:44:04.028ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "cd2195aa-9efc-4e6f-a02b-db932f165815", "content-length": "109", "date": "Wed, 30 Apr 2025 06:44:03 GMT"} })
72252025-04-30T06:44:04.028ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1/component/rot/caboose?firmware_slot=0
72262025-04-30T06:44:04.030ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "009b1cbe-0950-42bc-aa90-a047f8c99606", "content-length": "169", "date": "Wed, 30 Apr 2025 06:44:03 GMT"} })
72272025-04-30T06:44:04.030ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1/component/rot/caboose?firmware_slot=1
72282025-04-30T06:44:04.032ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "50483b70-ac93-4e6e-8263-b9cdfd799aa7", "content-length": "169", "date": "Wed, 30 Apr 2025 06:44:03 GMT"} })
72292025-04-30T06:44:04.033ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1/component/stage0/caboose?firmware_slot=0
72302025-04-30T06:44:04.035ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "5bf216d2-7135-425f-b08f-8740e96990d0", "content-length": "178", "date": "Wed, 30 Apr 2025 06:44:03 GMT"} })
72312025-04-30T06:44:04.035ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1/component/stage0/caboose?firmware_slot=1
72322025-04-30T06:44:04.037ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "48ef7efb-9574-444f-a71b-82031a7dfd93", "content-length": "178", "date": "Wed, 30 Apr 2025 06:44:03 GMT"} })
72332025-04-30T06:44:04.037ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1/component/rot/cmpa
72342025-04-30T06:44:04.038ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "10a6e3a3-165b-43c2-aa27-866063587148", "content-length": "702", "date": "Wed, 30 Apr 2025 06:44:03 GMT"} })
72352025-04-30T06:44:04.038ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1/component/rot/cfpa
72362025-04-30T06:44:04.039ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "29bec542-955c-41bc-b02b-bbca7919f016", "content-length": "727", "date": "Wed, 30 Apr 2025 06:44:03 GMT"} })
72372025-04-30T06:44:04.039ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1/component/rot/cfpa
72382025-04-30T06:44:04.040ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "ce8029d5-627f-44d3-8f8f-2ff7300d2fdd", "content-length": "729", "date": "Wed, 30 Apr 2025 06:44:03 GMT"} })
72392025-04-30T06:44:04.041ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1/component/rot/cfpa
72402025-04-30T06:44:04.042ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "64e9a568-ea2f-4f9b-8799-787b20c123fa", "content-length": "728", "date": "Wed, 30 Apr 2025 06:44:03 GMT"} })
72412025-04-30T06:44:04.042ZTRACtest_utilization (BackgroundTasks): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet
72422025-04-30T06:44:04.042ZTRACtest_utilization (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
72432025-04-30T06:44:04.042ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
72442025-04-30T06:44:04.042ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
72452025-04-30T06:44:04.043ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
72462025-04-30T06:44:04.049ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
72472025-04-30T06:44:04.053ZDEBGtest_utilization (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet result = Ok(())
72482025-04-30T06:44:04.053ZTRACtest_utilization (BackgroundTasks): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet
72492025-04-30T06:44:04.053ZTRACtest_utilization (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
72502025-04-30T06:44:04.053ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
72512025-04-30T06:44:04.053ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
72522025-04-30T06:44:04.053ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
72532025-04-30T06:44:04.076ZTRACtest_utilization (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory
72542025-04-30T06:44:04.076ZTRACtest_utilization (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
72552025-04-30T06:44:04.076ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
72562025-04-30T06:44:04.076ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
72572025-04-30T06:44:04.078ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
72582025-04-30T06:44:04.079ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
72592025-04-30T06:44:04.089ZDEBGtest_utilization (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet result = Ok(())
72602025-04-30T06:44:04.089ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
72612025-04-30T06:44:04.089ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
72622025-04-30T06:44:04.090ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
72632025-04-30T06:44:04.093ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
72642025-04-30T06:44:04.098ZDEBGtest_utilization (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory result = Ok(())
72652025-04-30T06:44:04.099ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
72662025-04-30T06:44:04.099ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
72672025-04-30T06:44:04.100ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
72682025-04-30T06:44:04.146ZDEBGtest_utilization (BackgroundTasks): begin collection from Sled Agent background_task = inventory_collection sled_agent_url = http://[::1]:39365
72692025-04-30T06:44:04.146ZDEBGtest_utilization (BackgroundTasks): client request SledAgent = http://[::1]:39365 background_task = inventory_collection body = None method = GET uri = http://[::1]:39365/inventory
72702025-04-30T06:44:04.148ZINFOtest_utilization (dropshot): accepted connection local_addr = [::1]:39365 remote_addr = [::1]:45382 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
72712025-04-30T06:44:04.148ZDEBGtest_utilization (collection-task): sent timer-based collection request to the collection task collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = ac55d8d9-a298-4534-898b-beee1c9e4e52
72722025-04-30T06:44:04.148ZDEBGtest_utilization (collection-task): collecting from producer collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = ac55d8d9-a298-4534-898b-beee1c9e4e52
72732025-04-30T06:44:04.149ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
72742025-04-30T06:44:04.149ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
72752025-04-30T06:44:04.151ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
72762025-04-30T06:44:04.151ZINFOtest_utilization (dropshot): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:1025 local_addr = 127.0.0.1:57688 remote_addr = 127.0.0.1:55243
72772025-04-30T06:44:04.151ZTRACtest_utilization (dropshot): incoming request local_addr = [::1]:39365 method = GET remote_addr = [::1]:45382 req_id = d7f2ea20-0ffa-47d0-8bad-7d016bf7dae3 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /inventory
72782025-04-30T06:44:04.152ZINFOtest_utilization (dropshot): request completed latency_us = 783 local_addr = [::1]:39365 method = GET remote_addr = [::1]:45382 req_id = d7f2ea20-0ffa-47d0-8bad-7d016bf7dae3 response_code = 200 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /inventory
72792025-04-30T06:44:04.152ZTRACtest_utilization (dropshot): incoming request local_addr = 127.0.0.1:57688 method = GET remote_addr = 127.0.0.1:55243 req_id = 33295b23-cd0a-4736-b98e-b57abc187f65 uri = /ac55d8d9-a298-4534-898b-beee1c9e4e52
72802025-04-30T06:44:04.153ZDEBGtest_utilization (BackgroundTasks): client response SledAgent = http://[::1]:39365 background_task = inventory_collection result = Ok(Response { url: "http://[::1]:39365/inventory", status: 200, headers: {"content-type": "application/json", "x-request-id": "d7f2ea20-0ffa-47d0-8bad-7d016bf7dae3", "content-length": "3141", "date": "Wed, 30 Apr 2025 06:44:03 GMT"} })
72812025-04-30T06:44:04.158ZDEBGtest_utilization (BackgroundTasks): begin collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
72822025-04-30T06:44:04.158ZDEBGtest_utilization (BackgroundTasks): end collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
72832025-04-30T06:44:04.158ZDEBGtest_utilization (BackgroundTasks): finished collection background_task = inventory_collection
72842025-04-30T06:44:04.161ZTRACtest_utilization (BackgroundTasks): authorize begin action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory
72852025-04-30T06:44:04.161ZTRACtest_utilization (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
72862025-04-30T06:44:04.161ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
72872025-04-30T06:44:04.162ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
72882025-04-30T06:44:04.162ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
72892025-04-30T06:44:04.176ZINFOtest_utilization (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 23473 local_addr = 127.0.0.1:57688 method = GET remote_addr = 127.0.0.1:55243 req_id = 33295b23-cd0a-4736-b98e-b57abc187f65 response_code = 200 uri = /ac55d8d9-a298-4534-898b-beee1c9e4e52
72902025-04-30T06:44:04.180ZTRACtest_utilization (collection-task): sent collection request to producer collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = ac55d8d9-a298-4534-898b-beee1c9e4e52
72912025-04-30T06:44:04.221ZDEBGtest_utilization (collection-task): collected results from producer collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 n_results = 5 producer_id = ac55d8d9-a298-4534-898b-beee1c9e4e52
72922025-04-30T06:44:04.221ZTRACtest_utilization (collection-task): forwarded results to main collection loop collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = ac55d8d9-a298-4534-898b-beee1c9e4e52
72932025-04-30T06:44:04.221ZTRACtest_utilization (collection-task): top of inner collection loop, waiting for next request collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = ac55d8d9-a298-4534-898b-beee1c9e4e52
72942025-04-30T06:44:04.221ZDEBGtest_utilization (results-sink): inserting 36 samples into database collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
72952025-04-30T06:44:04.221ZDEBGtest_utilization (results-sink-cluster): inserting 36 samples into database collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
72962025-04-30T06:44:04.221ZDEBGtest_utilization (clickhouse-client): unrolling 36 total samples collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 4d1485f0-110e-4161-9a33-c4610d68be5d
72972025-04-30T06:44:04.223ZDEBGtest_utilization (clickhouse-client): retrieving timeseries schema from database collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 4d1485f0-110e-4161-9a33-c4610d68be5d
72982025-04-30T06:44:04.223ZTRACtest_utilization (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 4d1485f0-110e-4161-9a33-c4610d68be5d sql = SELECT * FROM oximeter.timeseries_schema WHERE timeseries_name NOT IN ('integration_target:integration_metric') FORMAT Native;
72992025-04-30T06:44:04.223ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
73002025-04-30T06:44:04.229ZDEBGtest_utilization (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory result = Ok(())
73012025-04-30T06:44:04.229ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
73022025-04-30T06:44:04.229ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
73032025-04-30T06:44:04.230ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
73042025-04-30T06:44:04.232ZTRACtest_utilization (clickhouse-client): no new timeseries schema in database collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 4d1485f0-110e-4161-9a33-c4610d68be5d
73052025-04-30T06:44:04.232ZDEBGtest_utilization (clickhouse-client): new timeseries schema collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 4d1485f0-110e-4161-9a33-c4610d68be5d schema = TimeseriesSchema { timeseries_name: TimeseriesName("http_service:request_latency_histogram"), description: TimeseriesDescription { target: "", metric: "" }, field_schema: {FieldSchema { name: "id", field_type: Uuid, source: Target, description: "" }, FieldSchema { name: "name", field_type: String, source: Target, description: "" }, FieldSchema { name: "operation_id", field_type: String, source: Metric, description: "" }, FieldSchema { name: "status_code", field_type: U16, source: Metric, description: "" }}, datum_type: HistogramU64, version: 1, authz_scope: Fleet, units: Count, created: 2025-04-30T06:44:04.223301063Z } timeseries_name = http_service:request_latency_histogram
73062025-04-30T06:44:04.264ZDEBGtest_utilization (clickhouse-client): retrieving timeseries schema from database collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 4d1485f0-110e-4161-9a33-c4610d68be5d
73072025-04-30T06:44:04.264ZTRACtest_utilization (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 4d1485f0-110e-4161-9a33-c4610d68be5d sql = SELECT * FROM oximeter.timeseries_schema WHERE timeseries_name NOT IN ('http_service:request_latency_histogram', 'integration_target:integration_metric') FORMAT Native;
73082025-04-30T06:44:04.268ZTRACtest_utilization (clickhouse-client): no new timeseries schema in database collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 4d1485f0-110e-4161-9a33-c4610d68be5d
73092025-04-30T06:44:04.268ZDEBGtest_utilization (clickhouse-client): new timeseries schema collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 4d1485f0-110e-4161-9a33-c4610d68be5d schema = TimeseriesSchema { timeseries_name: TimeseriesName("collection_target:cpus_provisioned"), description: TimeseriesDescription { target: "", metric: "" }, field_schema: {FieldSchema { name: "id", field_type: Uuid, source: Target, description: "" }}, datum_type: I64, version: 1, authz_scope: Fleet, units: Count, created: 2025-04-30T06:44:04.264463446Z } timeseries_name = collection_target:cpus_provisioned
73102025-04-30T06:44:04.269ZDEBGtest_utilization (clickhouse-client): retrieving timeseries schema from database collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 4d1485f0-110e-4161-9a33-c4610d68be5d
73112025-04-30T06:44:04.269ZTRACtest_utilization (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 4d1485f0-110e-4161-9a33-c4610d68be5d sql = SELECT * FROM oximeter.timeseries_schema WHERE timeseries_name NOT IN ('collection_target:cpus_provisioned', 'http_service:request_latency_histogram', 'integration_target:integration_metric') FORMAT Native;
73122025-04-30T06:44:04.272ZTRACtest_utilization (clickhouse-client): no new timeseries schema in database collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 4d1485f0-110e-4161-9a33-c4610d68be5d
73132025-04-30T06:44:04.272ZDEBGtest_utilization (clickhouse-client): new timeseries schema collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 4d1485f0-110e-4161-9a33-c4610d68be5d schema = TimeseriesSchema { timeseries_name: TimeseriesName("collection_target:ram_provisioned"), description: TimeseriesDescription { target: "", metric: "" }, field_schema: {FieldSchema { name: "id", field_type: Uuid, source: Target, description: "" }}, datum_type: I64, version: 1, authz_scope: Fleet, units: Count, created: 2025-04-30T06:44:04.269625812Z } timeseries_name = collection_target:ram_provisioned
73142025-04-30T06:44:04.273ZDEBGtest_utilization (clickhouse-client): retrieving timeseries schema from database collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 4d1485f0-110e-4161-9a33-c4610d68be5d
73152025-04-30T06:44:04.273ZTRACtest_utilization (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 4d1485f0-110e-4161-9a33-c4610d68be5d sql = SELECT * FROM oximeter.timeseries_schema WHERE timeseries_name NOT IN ('collection_target:cpus_provisioned', 'collection_target:ram_provisioned', 'http_service:request_latency_histogram', 'integration_target:integration_metric') FORMAT Native;
73162025-04-30T06:44:04.276ZTRACtest_utilization (clickhouse-client): no new timeseries schema in database collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 4d1485f0-110e-4161-9a33-c4610d68be5d
73172025-04-30T06:44:04.276ZDEBGtest_utilization (clickhouse-client): new timeseries schema collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 4d1485f0-110e-4161-9a33-c4610d68be5d schema = TimeseriesSchema { timeseries_name: TimeseriesName("collection_target:virtual_disk_space_provisioned"), description: TimeseriesDescription { target: "", metric: "" }, field_schema: {FieldSchema { name: "id", field_type: Uuid, source: Target, description: "" }}, datum_type: I64, version: 1, authz_scope: Fleet, units: Count, created: 2025-04-30T06:44:04.273304121Z } timeseries_name = collection_target:virtual_disk_space_provisioned
73182025-04-30T06:44:04.293ZDEBGtest_utilization (clickhouse-client): inserting 4 new timeseries schema collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 4d1485f0-110e-4161-9a33-c4610d68be5d
73192025-04-30T06:44:04.293ZTRACtest_utilization (clickhouse-client): inserting data collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 4d1485f0-110e-4161-9a33-c4610d68be5d n_columns = 6 n_rows = 4 sql = INSERT INTO oximeter.timeseries_schema FORMAT Native
73202025-04-30T06:44:04.298ZTRACtest_utilization (clickhouse-client): inserting data collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 4d1485f0-110e-4161-9a33-c4610d68be5d n_columns = 4 n_rows = 30 sql = INSERT INTO oximeter.fields_string FORMAT Native
73212025-04-30T06:44:04.302ZDEBGtest_utilization (clickhouse-client): inserted rows into table collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 4d1485f0-110e-4161-9a33-c4610d68be5d n_rows = 30 table_name = fields_string
73222025-04-30T06:44:04.302ZTRACtest_utilization (clickhouse-client): inserting data collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 4d1485f0-110e-4161-9a33-c4610d68be5d n_columns = 4 n_rows = 14 sql = INSERT INTO oximeter.fields_u16 FORMAT Native
73232025-04-30T06:44:04.306ZDEBGtest_utilization (clickhouse-client): inserted rows into table collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 4d1485f0-110e-4161-9a33-c4610d68be5d n_rows = 14 table_name = fields_u16
73242025-04-30T06:44:04.306ZTRACtest_utilization (clickhouse-client): inserting data collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 4d1485f0-110e-4161-9a33-c4610d68be5d n_columns = 4 n_rows = 29 sql = INSERT INTO oximeter.fields_uuid FORMAT Native
73252025-04-30T06:44:04.310ZDEBGtest_utilization (clickhouse-client): inserted rows into table collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 4d1485f0-110e-4161-9a33-c4610d68be5d n_rows = 29 table_name = fields_uuid
73262025-04-30T06:44:04.310ZTRACtest_utilization (clickhouse-client): inserting data collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 4d1485f0-110e-4161-9a33-c4610d68be5d n_columns = 19 n_rows = 14 sql = INSERT INTO oximeter.measurements_histogramu64 FORMAT Native
73272025-04-30T06:44:04.318ZDEBGtest_utilization (clickhouse-client): inserted rows into table collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 4d1485f0-110e-4161-9a33-c4610d68be5d n_rows = 14 table_name = measurements_histogramu64
73282025-04-30T06:44:04.318ZTRACtest_utilization (clickhouse-client): inserting data collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 4d1485f0-110e-4161-9a33-c4610d68be5d n_columns = 4 n_rows = 22 sql = INSERT INTO oximeter.measurements_i64 FORMAT Native
73292025-04-30T06:44:04.319ZWARNtest_utilization (datastore): Non-retryable transaction failure err = DatabaseError(UniqueViolation, "duplicate key value violates unique constraint \\"inv_caboose_pkey\\"") transaction = inventory_insert_collection
73302025-04-30T06:44:04.319ZWARNtest_utilization (BackgroundTasks): inventory collection failed background_task = inventory_collection error = failed to collect inventory: saving inventory to database: Internal Error: unexpected database error: duplicate key value violates unique constraint "inv_caboose_pkey"
73312025-04-30T06:44:04.319ZDEBGtest_utilization (BackgroundTasks): activation complete background_task = inventory_collection elapsed = 611.461025ms iteration = 9 status = {"error":"failed to collect inventory: saving inventory to database: Internal Error: unexpected database error: duplicate key value violates unique constraint \\"inv_caboose_pkey\\""}
73322025-04-30T06:44:04.319ZDEBGtest_utilization (BackgroundTasks): activating background_task = inventory_collection iteration = 10 reason = Signaled
73332025-04-30T06:44:04.319ZTRACtest_utilization (BackgroundTasks): authorize begin action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory
73342025-04-30T06:44:04.319ZTRACtest_utilization (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
73352025-04-30T06:44:04.319ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
73362025-04-30T06:44:04.319ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
73372025-04-30T06:44:04.320ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
73382025-04-30T06:44:04.321ZDEBGtest_utilization: attempt to fetch latest inventory collection result = Ok("found")
73392025-04-30T06:44:04.323ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
73402025-04-30T06:44:04.329ZDEBGtest_utilization (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory result = Ok(())
73412025-04-30T06:44:04.329ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
73422025-04-30T06:44:04.329ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
73432025-04-30T06:44:04.330ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
73442025-04-30T06:44:04.330ZDEBGtest_utilization (clickhouse-client): inserted rows into table collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 4d1485f0-110e-4161-9a33-c4610d68be5d n_rows = 22 table_name = measurements_i64
73452025-04-30T06:44:04.330ZTRACtest_utilization (results-sink): successfully inserted samples collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
73462025-04-30T06:44:04.330ZWARNtest_utilization (results-sink-cluster): failed to insert some results into metric DB collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 error = Failed to check out connection to database: No backends found for this service file = oximeter/collector/src/results_sink.rs:92
73472025-04-30T06:44:04.333ZDEBGtest_utilization (BackgroundTasks): inventory_prune_one: nothing eligible for removal (too few) background_task = inventory_collection candidates = [(13d8b8d9-6c6b-47a6-9190-952544edb75b, 0)]
73482025-04-30T06:44:04.333ZTRACtest_utilization (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal
73492025-04-30T06:44:04.333ZTRACtest_utilization (dns): buffer buffer = Length: 67 (0x43) bytes\n0000: 0b 49 01 00 00 01 00 00 00 00 00 01 04 5f 6d 67 .I..........._mg\n0010: 73 04 5f 74 63 70 0d 63 6f 6e 74 72 6f 6c 2d 70 s._tcp.control-p\n0020: 6c 61 6e 65 05 6f 78 69 64 65 08 69 6e 74 65 72 lane.oxide.inter\n0030: 6e 61 6c 00 00 21 00 01 00 00 29 04 d0 00 00 00 nal..!....).....\n0040: 00 00 00 ... kind = dns peer_addr = [::1]:52708 req_id = 24e50852-9cb9-4532-b1ed-ab4d5af16711
73502025-04-30T06:44:04.333ZDEBGtest_utilization (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 2889,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_mgs._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_mgs._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 4,\n 95,\n 109,\n 103,\n 115,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:52708 req_id = 24e50852-9cb9-4532-b1ed-ab4d5af16711
73512025-04-30T06:44:04.334ZDEBGtest_utilization (store): query key key = _mgs._tcp
73522025-04-30T06:44:04.334ZDEBGtest_utilization (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
73532025-04-30T06:44:04.334ZDEBGtest_utilization (dns): dns response additional_records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal"), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] kind = dns peer_addr = [::1]:52708 query = LowerQuery { name: LowerName(Name("_mgs._tcp.control-plane.oxide.internal.")), original: Query { name: Name("_mgs._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN } } records = [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 48759, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal") })) }] req_id = 24e50852-9cb9-4532-b1ed-ab4d5af16711
73542025-04-30T06:44:04.334ZDEBGtest_utilization (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgs._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 48759, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }], valid_until: Instant { tv_sec: 4309, tv_nsec: 133635044 } })
73552025-04-30T06:44:04.334ZTRACtest_utilization (DnsResolver): lookup_service_targets: looking up SRV target
73562025-04-30T06:44:04.335ZTRACtest_utilization (dns): buffer buffer = Length: 108 (0x6c) bytes\n0000: df 4f 01 00 00 01 00 00 00 00 00 01 2d 64 65 6e .O..........-den\n0010: 64 72 69 74 65 2d 62 36 64 36 35 33 34 31 2d 31 drite-b6d65341-1\n0020: 36 37 63 2d 34 31 64 66 2d 39 62 35 63 2d 34 31 67c-41df-9b5c-41\n0030: 63 64 65 64 39 39 63 32 32 39 04 68 6f 73 74 0d cded99c229.host.\n0040: 63 6f 6e 74 72 6f 6c 2d 70 6c 61 6e 65 05 6f 78 control-plane.ox\n0050: 69 64 65 08 69 6e 74 65 72 6e 61 6c 00 00 1c 00 ide.internal....\n0060: 01 00 00 29 04 d0 00 00 00 00 00 00 ...)........ kind = dns peer_addr = [::1]:50755 req_id = da8471f6-3e28-4463-9cf1-272e77d8a5e9
73572025-04-30T06:44:04.335ZDEBGtest_utilization (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 57167,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n query_type: AAAA,\n query_class: IN,\n },\n },\n original: [\n 45,\n 100,\n 101,\n 110,\n 100,\n 114,\n 105,\n 116,\n 101,\n 45,\n 98,\n 54,\n 100,\n 54,\n 53,\n 51,\n 52,\n 49,\n 45,\n 49,\n 54,\n 55,\n 99,\n 45,\n 52,\n 49,\n 100,\n 102,\n 45,\n 57,\n 98,\n 53,\n 99,\n 45,\n 52,\n 49,\n 99,\n 100,\n 101,\n 100,\n 57,\n 57,\n 99,\n 50,\n 50,\n 57,\n 4,\n 104,\n 111,\n 115,\n 116,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 28,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:50755 req_id = da8471f6-3e28-4463-9cf1-272e77d8a5e9
73582025-04-30T06:44:04.336ZDEBGtest_utilization (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
73592025-04-30T06:44:04.336ZDEBGtest_utilization (dns): dns response additional_records = [] kind = dns peer_addr = [::1]:50755 query = LowerQuery { name: LowerName(Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.")), original: Query { name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), query_type: AAAA, query_class: IN } } records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] req_id = da8471f6-3e28-4463-9cf1-272e77d8a5e9
73602025-04-30T06:44:04.403ZTRACtest_utilization (DnsResolver): lookup_all_socket_v6 srv dns_name = _clickhouse-admin-keeper._tcp.control-plane.oxide.internal
73612025-04-30T06:44:04.404ZTRACtest_utilization (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory
73622025-04-30T06:44:04.404ZTRACtest_utilization (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
73632025-04-30T06:44:04.404ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
73642025-04-30T06:44:04.404ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
73652025-04-30T06:44:04.405ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
73662025-04-30T06:44:04.406ZTRACtest_utilization (results-sink): batch interval expired, but no samples to insert collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
73672025-04-30T06:44:04.406ZTRACtest_utilization (results-sink-cluster): batch interval expired, but no samples to insert collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
73682025-04-30T06:44:04.407ZTRACtest_utilization (dns): buffer buffer = Length: 87 (0x57) bytes\n0000: 39 95 01 00 00 01 00 00 00 00 00 01 18 5f 63 6c 9............_cl\n0010: 69 63 6b 68 6f 75 73 65 2d 61 64 6d 69 6e 2d 6b ickhouse-admin-k\n0020: 65 65 70 65 72 04 5f 74 63 70 0d 63 6f 6e 74 72 eeper._tcp.contr\n0030: 6f 6c 2d 70 6c 61 6e 65 05 6f 78 69 64 65 08 69 ol-plane.oxide.i\n0040: 6e 74 65 72 6e 61 6c 00 00 21 00 01 00 00 29 04 nternal..!....).\n0050: d0 00 00 00 00 00 00 ....... kind = dns peer_addr = [::1]:63492 req_id = 3fd89bd1-47bc-484a-a768-ec1ded606874
73692025-04-30T06:44:04.407ZDEBGtest_utilization (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 14741,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 24,\n 95,\n 99,\n 108,\n 105,\n 99,\n 107,\n 104,\n 111,\n 117,\n 115,\n 101,\n 45,\n 97,\n 100,\n 109,\n 105,\n 110,\n 45,\n 107,\n 101,\n 101,\n 112,\n 101,\n 114,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:63492 req_id = 3fd89bd1-47bc-484a-a768-ec1ded606874
73702025-04-30T06:44:04.407ZDEBGtest_utilization (store): query key key = _clickhouse-admin-keeper._tcp
73712025-04-30T06:44:04.407ZERROtest_utilization (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 14741, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 1, }, query: WireQuery { query: LowerQuery { name: LowerName( Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), ), original: Query { name: Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN, }, }, original: [ 24, 95, 99, 108, 105, 99, 107, 104, 111, 117, 115, 101, 45, 97, 100, 109, 105, 110, 45, 107, 101, 101, 112, 101, 114, 4, 95, 116, 99, 112, 13, 99, 111, 110, 116, 114, 111, 108, 45, 112, 108, 97, 110, 101, 5, 111, 120, 105, 100, 101, 8, 105, 110, 116, 101, 114, 110, 97, 108, 0, 0, 33, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: Some( Edns { rcode_high: 0, version: 0, dnssec_ok: false, max_payload: 1232, options: OPT { options: {}, }, }, ), } NXDOMAIN: no records found for name: "_clickhouse-admin-keeper._tcp.control-plane.oxide.internal." kind = dns peer_addr = [::1]:63492 req_id = 3fd89bd1-47bc-484a-a768-ec1ded606874
73722025-04-30T06:44:04.407ZDEBGtest_utilization (BackgroundTasks): begin collection background_task = inventory_collection
73732025-04-30T06:44:04.408ZDEBGtest_utilization (BackgroundTasks): begin collection from MGS background_task = inventory_collection mgs_url = http://[::1]:48759
73742025-04-30T06:44:04.408ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/ignition
73752025-04-30T06:44:04.410ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
73762025-04-30T06:44:04.422ZDEBGtest_utilization (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory result = Ok(())
73772025-04-30T06:44:04.422ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
73782025-04-30T06:44:04.422ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
73792025-04-30T06:44:04.423ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
73802025-04-30T06:44:04.426ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/ignition", status: 200, headers: {"content-type": "application/json", "x-request-id": "6ea7ed67-7bc1-4cd5-b3b0-09efe0b13516", "content-length": "835", "date": "Wed, 30 Apr 2025 06:44:04 GMT"} })
73812025-04-30T06:44:04.427ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0
73822025-04-30T06:44:04.428ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
73832025-04-30T06:44:04.429ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
73842025-04-30T06:44:04.429ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
73852025-04-30T06:44:04.433ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "0c6732da-d4ea-4f34-b5b8-b91cbb841252", "content-length": "734", "date": "Wed, 30 Apr 2025 06:44:04 GMT"} })
73862025-04-30T06:44:04.445ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0/component/sp/caboose?firmware_slot=0
73872025-04-30T06:44:04.464ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "ae7b93cb-57db-4616-9d8f-aa2c01c5be65", "content-length": "111", "date": "Wed, 30 Apr 2025 06:44:04 GMT"} })
73882025-04-30T06:44:04.464ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0/component/sp/caboose?firmware_slot=1
73892025-04-30T06:44:04.472ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "dd7ef5d8-a4f3-4a82-8190-f2a094a10b14", "content-length": "111", "date": "Wed, 30 Apr 2025 06:44:04 GMT"} })
73902025-04-30T06:44:04.472ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0/component/rot/caboose?firmware_slot=0
73912025-04-30T06:44:04.472ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "b9deeb90-5fbc-49f8-920a-961a8402b0c3", "content-length": "170", "date": "Wed, 30 Apr 2025 06:44:04 GMT"} })
73922025-04-30T06:44:04.472ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0/component/rot/caboose?firmware_slot=1
73932025-04-30T06:44:04.482ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "d239f8ce-464f-4fa0-beab-0abf4bc6eb68", "content-length": "170", "date": "Wed, 30 Apr 2025 06:44:04 GMT"} })
73942025-04-30T06:44:04.482ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0/component/stage0/caboose?firmware_slot=0
73952025-04-30T06:44:04.482ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "aeb17d57-86dc-409c-b689-e62f3bf87c2f", "content-length": "179", "date": "Wed, 30 Apr 2025 06:44:04 GMT"} })
73962025-04-30T06:44:04.482ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0/component/stage0/caboose?firmware_slot=1
73972025-04-30T06:44:04.484ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "cb74c8bd-e35e-43a2-b21d-0a6ee3094fa2", "content-length": "179", "date": "Wed, 30 Apr 2025 06:44:04 GMT"} })
73982025-04-30T06:44:04.485ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0/component/rot/cmpa
73992025-04-30T06:44:04.487ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "ca7c4e45-bb9c-435d-b68f-a19ea4094ad9", "content-length": "702", "date": "Wed, 30 Apr 2025 06:44:04 GMT"} })
74002025-04-30T06:44:04.487ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0/component/rot/cfpa
74012025-04-30T06:44:04.490ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "45467c91-7047-49b5-8e05-3ac535e08b23", "content-length": "727", "date": "Wed, 30 Apr 2025 06:44:04 GMT"} })
74022025-04-30T06:44:04.490ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0/component/rot/cfpa
74032025-04-30T06:44:04.491ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "17dabd54-2e0d-42b6-8ce9-8fbf5cde2f3f", "content-length": "729", "date": "Wed, 30 Apr 2025 06:44:04 GMT"} })
74042025-04-30T06:44:04.492ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0/component/rot/cfpa
74052025-04-30T06:44:04.496ZDEBGtest_utilization: attempt to fetch latest inventory collection result = Ok("found")
74062025-04-30T06:44:04.498ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "c1c2976b-b2b0-44a6-bf31-94057f1e9e9f", "content-length": "728", "date": "Wed, 30 Apr 2025 06:44:04 GMT"} })
74072025-04-30T06:44:04.498ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1
74082025-04-30T06:44:04.502ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1", status: 200, headers: {"content-type": "application/json", "x-request-id": "1bb5493a-3df6-422a-b939-99c1de927889", "content-length": "734", "date": "Wed, 30 Apr 2025 06:44:04 GMT"} })
74092025-04-30T06:44:04.502ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1/component/sp/caboose?firmware_slot=0
74102025-04-30T06:44:04.510ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "7b9e6fbf-8226-4177-b5da-ba7afccabbaa", "content-length": "111", "date": "Wed, 30 Apr 2025 06:44:04 GMT"} })
74112025-04-30T06:44:04.511ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1/component/sp/caboose?firmware_slot=1
74122025-04-30T06:44:04.514ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "f8c8608d-0a2e-46c7-ac29-3dd04d246d48", "content-length": "111", "date": "Wed, 30 Apr 2025 06:44:04 GMT"} })
74132025-04-30T06:44:04.515ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1/component/rot/caboose?firmware_slot=0
74142025-04-30T06:44:04.518ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "e20b5e1b-f332-4ea6-a7a8-0119337217f8", "content-length": "170", "date": "Wed, 30 Apr 2025 06:44:04 GMT"} })
74152025-04-30T06:44:04.518ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1/component/rot/caboose?firmware_slot=1
74162025-04-30T06:44:04.522ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "9d678b78-4886-44ce-bb28-a78e5cc9af44", "content-length": "170", "date": "Wed, 30 Apr 2025 06:44:04 GMT"} })
74172025-04-30T06:44:04.522ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1/component/stage0/caboose?firmware_slot=0
74182025-04-30T06:44:04.525ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "9623b6ff-b0cf-479a-a3c2-fdc4570c2f53", "content-length": "179", "date": "Wed, 30 Apr 2025 06:44:04 GMT"} })
74192025-04-30T06:44:04.525ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1/component/stage0/caboose?firmware_slot=1
74202025-04-30T06:44:04.527ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "b1864e8d-4716-4d3a-ac9a-3a41add5c8e2", "content-length": "179", "date": "Wed, 30 Apr 2025 06:44:04 GMT"} })
74212025-04-30T06:44:04.527ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1/component/rot/cmpa
74222025-04-30T06:44:04.528ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "ca7abd00-1abb-41e7-9769-3209704bc43f", "content-length": "702", "date": "Wed, 30 Apr 2025 06:44:04 GMT"} })
74232025-04-30T06:44:04.528ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1/component/rot/cfpa
74242025-04-30T06:44:04.530ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "74a67f55-7ebf-4081-a0a3-b9b5ac4d8851", "content-length": "727", "date": "Wed, 30 Apr 2025 06:44:04 GMT"} })
74252025-04-30T06:44:04.530ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1/component/rot/cfpa
74262025-04-30T06:44:04.531ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "dee50cf9-87e6-402a-84f6-83480335cb81", "content-length": "729", "date": "Wed, 30 Apr 2025 06:44:04 GMT"} })
74272025-04-30T06:44:04.531ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1/component/rot/cfpa
74282025-04-30T06:44:04.532ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "0b382016-e8dc-4c57-ae12-1b8ea7545eea", "content-length": "728", "date": "Wed, 30 Apr 2025 06:44:04 GMT"} })
74292025-04-30T06:44:04.533ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0
74302025-04-30T06:44:04.534ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "1d9d9ca2-ddd8-4948-87b0-c3e080e629c8", "content-length": "723", "date": "Wed, 30 Apr 2025 06:44:04 GMT"} })
74312025-04-30T06:44:04.535ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0/component/sp/caboose?firmware_slot=0
74322025-04-30T06:44:04.538ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "3e2a0045-6ebd-48b3-8aa5-5d68e75094aa", "content-length": "109", "date": "Wed, 30 Apr 2025 06:44:04 GMT"} })
74332025-04-30T06:44:04.538ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0/component/sp/caboose?firmware_slot=1
74342025-04-30T06:44:04.543ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "101d55d8-799a-490c-926e-b610485679de", "content-length": "109", "date": "Wed, 30 Apr 2025 06:44:04 GMT"} })
74352025-04-30T06:44:04.543ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0/component/rot/caboose?firmware_slot=0
74362025-04-30T06:44:04.546ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "7b3ee8ff-db21-410e-a75d-afb193a5ecc7", "content-length": "169", "date": "Wed, 30 Apr 2025 06:44:04 GMT"} })
74372025-04-30T06:44:04.547ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0/component/rot/caboose?firmware_slot=1
74382025-04-30T06:44:04.547ZTRACtest_utilization (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory
74392025-04-30T06:44:04.547ZTRACtest_utilization (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
74402025-04-30T06:44:04.547ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
74412025-04-30T06:44:04.547ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
74422025-04-30T06:44:04.549ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
74432025-04-30T06:44:04.553ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
74442025-04-30T06:44:04.562ZDEBGtest_utilization (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory result = Ok(())
74452025-04-30T06:44:04.562ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
74462025-04-30T06:44:04.562ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
74472025-04-30T06:44:04.563ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
74482025-04-30T06:44:04.564ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "4d2e66b4-378e-4453-a285-211798b8edab", "content-length": "169", "date": "Wed, 30 Apr 2025 06:44:04 GMT"} })
74492025-04-30T06:44:04.564ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0/component/stage0/caboose?firmware_slot=0
74502025-04-30T06:44:04.566ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
74512025-04-30T06:44:04.566ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
74522025-04-30T06:44:04.568ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
74532025-04-30T06:44:04.644ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "3b39533e-924c-4e30-aa7d-ef8f1be1e80b", "content-length": "178", "date": "Wed, 30 Apr 2025 06:44:04 GMT"} })
74542025-04-30T06:44:04.644ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0/component/stage0/caboose?firmware_slot=1
74552025-04-30T06:44:04.659ZDEBGtest_utilization: attempt to fetch latest inventory collection result = Ok("found")
74562025-04-30T06:44:04.662ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "2a905712-2dc0-43d9-9dda-a582c032c38d", "content-length": "178", "date": "Wed, 30 Apr 2025 06:44:04 GMT"} })
74572025-04-30T06:44:04.663ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0/component/rot/cmpa
74582025-04-30T06:44:04.664ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "cc872051-9a9a-4596-abd9-bdb5edc84648", "content-length": "702", "date": "Wed, 30 Apr 2025 06:44:04 GMT"} })
74592025-04-30T06:44:04.665ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0/component/rot/cfpa
74602025-04-30T06:44:04.666ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "606e29b0-8a8a-45ee-ad5e-a8287da7c0c8", "content-length": "727", "date": "Wed, 30 Apr 2025 06:44:04 GMT"} })
74612025-04-30T06:44:04.666ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0/component/rot/cfpa
74622025-04-30T06:44:04.668ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "7bc3b52c-14bd-4dea-843b-49cf27b7a4d7", "content-length": "729", "date": "Wed, 30 Apr 2025 06:44:04 GMT"} })
74632025-04-30T06:44:04.668ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0/component/rot/cfpa
74642025-04-30T06:44:04.669ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "75c719e2-254d-4dcd-a724-fdbfa1b6fc68", "content-length": "728", "date": "Wed, 30 Apr 2025 06:44:04 GMT"} })
74652025-04-30T06:44:04.669ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1
74662025-04-30T06:44:04.671ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1", status: 200, headers: {"content-type": "application/json", "x-request-id": "92fd95d0-4dfb-430b-85ff-a7f26c0a387d", "content-length": "723", "date": "Wed, 30 Apr 2025 06:44:04 GMT"} })
74672025-04-30T06:44:04.671ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1/component/sp/caboose?firmware_slot=0
74682025-04-30T06:44:04.673ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "04519f22-94c5-458e-b8c1-e43be1f14074", "content-length": "109", "date": "Wed, 30 Apr 2025 06:44:04 GMT"} })
74692025-04-30T06:44:04.673ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1/component/sp/caboose?firmware_slot=1
74702025-04-30T06:44:04.678ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "4553add3-2149-44f3-b811-520a1d529450", "content-length": "109", "date": "Wed, 30 Apr 2025 06:44:04 GMT"} })
74712025-04-30T06:44:04.678ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1/component/rot/caboose?firmware_slot=0
74722025-04-30T06:44:04.681ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "5a43edb8-e14a-4c2c-bb50-52e0c72e6c84", "content-length": "169", "date": "Wed, 30 Apr 2025 06:44:04 GMT"} })
74732025-04-30T06:44:04.681ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1/component/rot/caboose?firmware_slot=1
74742025-04-30T06:44:04.683ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "2e4d9538-5719-4e3f-810f-aa9d7032d11e", "content-length": "169", "date": "Wed, 30 Apr 2025 06:44:04 GMT"} })
74752025-04-30T06:44:04.683ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1/component/stage0/caboose?firmware_slot=0
74762025-04-30T06:44:04.685ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "ee9886cb-d741-4598-81c2-ee88ab656ea3", "content-length": "178", "date": "Wed, 30 Apr 2025 06:44:04 GMT"} })
74772025-04-30T06:44:04.686ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1/component/stage0/caboose?firmware_slot=1
74782025-04-30T06:44:04.687ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "ad956557-019f-4a6d-a90d-ea6f38e93388", "content-length": "178", "date": "Wed, 30 Apr 2025 06:44:04 GMT"} })
74792025-04-30T06:44:04.688ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1/component/rot/cmpa
74802025-04-30T06:44:04.689ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "23b2b7e6-006e-4595-9f0b-588632d2258a", "content-length": "702", "date": "Wed, 30 Apr 2025 06:44:04 GMT"} })
74812025-04-30T06:44:04.689ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1/component/rot/cfpa
74822025-04-30T06:44:04.690ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "c95cd899-4ca7-4a5f-91bf-db2b9749f70b", "content-length": "727", "date": "Wed, 30 Apr 2025 06:44:04 GMT"} })
74832025-04-30T06:44:04.690ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1/component/rot/cfpa
74842025-04-30T06:44:04.692ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "7f3866e3-2a3a-4278-b58b-0769f2a4bd13", "content-length": "729", "date": "Wed, 30 Apr 2025 06:44:04 GMT"} })
74852025-04-30T06:44:04.692ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1/component/rot/cfpa
74862025-04-30T06:44:04.693ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "3ab4de39-4e87-4f3c-ac84-9a306527e766", "content-length": "728", "date": "Wed, 30 Apr 2025 06:44:04 GMT"} })
74872025-04-30T06:44:04.694ZTRACtest_utilization (BackgroundTasks): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet
74882025-04-30T06:44:04.694ZTRACtest_utilization (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
74892025-04-30T06:44:04.694ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
74902025-04-30T06:44:04.694ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
74912025-04-30T06:44:04.695ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
74922025-04-30T06:44:04.698ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
74932025-04-30T06:44:04.706ZDEBGtest_utilization (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet result = Ok(())
74942025-04-30T06:44:04.706ZTRACtest_utilization (BackgroundTasks): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet
74952025-04-30T06:44:04.706ZTRACtest_utilization (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
74962025-04-30T06:44:04.706ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
74972025-04-30T06:44:04.706ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
74982025-04-30T06:44:04.707ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
74992025-04-30T06:44:04.710ZTRACtest_utilization (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory
75002025-04-30T06:44:04.710ZTRACtest_utilization (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
75012025-04-30T06:44:04.710ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
75022025-04-30T06:44:04.710ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
75032025-04-30T06:44:04.711ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
75042025-04-30T06:44:04.711ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
75052025-04-30T06:44:04.717ZDEBGtest_utilization (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet result = Ok(())
75062025-04-30T06:44:04.717ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
75072025-04-30T06:44:04.717ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
75082025-04-30T06:44:04.717ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
75092025-04-30T06:44:04.720ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
75102025-04-30T06:44:04.742ZDEBGtest_utilization (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory result = Ok(())
75112025-04-30T06:44:04.742ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
75122025-04-30T06:44:04.742ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
75132025-04-30T06:44:04.743ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
75142025-04-30T06:44:04.796ZDEBGtest_utilization (BackgroundTasks): begin collection from Sled Agent background_task = inventory_collection sled_agent_url = http://[::1]:39365
75152025-04-30T06:44:04.796ZDEBGtest_utilization (BackgroundTasks): client request SledAgent = http://[::1]:39365 background_task = inventory_collection body = None method = GET uri = http://[::1]:39365/inventory
75162025-04-30T06:44:04.797ZINFOtest_utilization (dropshot): accepted connection local_addr = [::1]:39365 remote_addr = [::1]:57152 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
75172025-04-30T06:44:04.799ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
75182025-04-30T06:44:04.799ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
75192025-04-30T06:44:04.800ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
75202025-04-30T06:44:04.800ZTRACtest_utilization (dropshot): incoming request local_addr = [::1]:39365 method = GET remote_addr = [::1]:57152 req_id = 6c21fa37-d232-4d0e-9da8-b374dcab0f54 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /inventory
75212025-04-30T06:44:04.801ZINFOtest_utilization (dropshot): request completed latency_us = 1079 local_addr = [::1]:39365 method = GET remote_addr = [::1]:57152 req_id = 6c21fa37-d232-4d0e-9da8-b374dcab0f54 response_code = 200 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /inventory
75222025-04-30T06:44:04.802ZDEBGtest_utilization (BackgroundTasks): client response SledAgent = http://[::1]:39365 background_task = inventory_collection result = Ok(Response { url: "http://[::1]:39365/inventory", status: 200, headers: {"content-type": "application/json", "x-request-id": "6c21fa37-d232-4d0e-9da8-b374dcab0f54", "content-length": "3141", "date": "Wed, 30 Apr 2025 06:44:04 GMT"} })
75232025-04-30T06:44:04.805ZDEBGtest_utilization (BackgroundTasks): begin collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
75242025-04-30T06:44:04.805ZDEBGtest_utilization (BackgroundTasks): end collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
75252025-04-30T06:44:04.805ZDEBGtest_utilization (BackgroundTasks): finished collection background_task = inventory_collection
75262025-04-30T06:44:04.809ZTRACtest_utilization (BackgroundTasks): authorize begin action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory
75272025-04-30T06:44:04.809ZTRACtest_utilization (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
75282025-04-30T06:44:04.809ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
75292025-04-30T06:44:04.809ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
75302025-04-30T06:44:04.809ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
75312025-04-30T06:44:04.811ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
75322025-04-30T06:44:04.815ZDEBGtest_utilization (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory result = Ok(())
75332025-04-30T06:44:04.816ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
75342025-04-30T06:44:04.816ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
75352025-04-30T06:44:04.816ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
75362025-04-30T06:44:04.850ZDEBGtest_utilization: attempt to fetch latest inventory collection result = Ok("found")
75372025-04-30T06:44:04.850ZWARNtest_utilization (datastore): Non-retryable transaction failure err = DatabaseError(UniqueViolation, "duplicate key value violates unique constraint \\"inv_caboose_pkey\\"") transaction = inventory_insert_collection
75382025-04-30T06:44:04.851ZWARNtest_utilization (BackgroundTasks): inventory collection failed background_task = inventory_collection error = failed to collect inventory: saving inventory to database: Internal Error: unexpected database error: duplicate key value violates unique constraint "inv_caboose_pkey"
75392025-04-30T06:44:04.851ZDEBGtest_utilization (BackgroundTasks): activation complete background_task = inventory_collection elapsed = 532.176987ms iteration = 10 status = {"error":"failed to collect inventory: saving inventory to database: Internal Error: unexpected database error: duplicate key value violates unique constraint \\"inv_caboose_pkey\\""}
75402025-04-30T06:44:04.851ZDEBGtest_utilization (BackgroundTasks): activating background_task = inventory_collection iteration = 11 reason = Signaled
75412025-04-30T06:44:04.851ZTRACtest_utilization (BackgroundTasks): authorize begin action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory
75422025-04-30T06:44:04.851ZTRACtest_utilization (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
75432025-04-30T06:44:04.851ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
75442025-04-30T06:44:04.851ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
75452025-04-30T06:44:04.852ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
75462025-04-30T06:44:04.854ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
75472025-04-30T06:44:04.859ZDEBGtest_utilization (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory result = Ok(())
75482025-04-30T06:44:04.859ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
75492025-04-30T06:44:04.859ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
75502025-04-30T06:44:04.860ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
75512025-04-30T06:44:04.863ZDEBGtest_utilization (BackgroundTasks): inventory_prune_one: nothing eligible for removal (too few) background_task = inventory_collection candidates = [(13d8b8d9-6c6b-47a6-9190-952544edb75b, 0)]
75522025-04-30T06:44:04.863ZTRACtest_utilization (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal
75532025-04-30T06:44:04.864ZTRACtest_utilization (dns): buffer buffer = Length: 67 (0x43) bytes\n0000: bc 75 01 00 00 01 00 00 00 00 00 01 04 5f 6d 67 .u..........._mg\n0010: 73 04 5f 74 63 70 0d 63 6f 6e 74 72 6f 6c 2d 70 s._tcp.control-p\n0020: 6c 61 6e 65 05 6f 78 69 64 65 08 69 6e 74 65 72 lane.oxide.inter\n0030: 6e 61 6c 00 00 21 00 01 00 00 29 04 d0 00 00 00 nal..!....).....\n0040: 00 00 00 ... kind = dns peer_addr = [::1]:58750 req_id = 0d1ec17b-ceb6-482e-9865-2697af6e480a
75542025-04-30T06:44:04.864ZDEBGtest_utilization (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 48245,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_mgs._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_mgs._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 4,\n 95,\n 109,\n 103,\n 115,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:58750 req_id = 0d1ec17b-ceb6-482e-9865-2697af6e480a
75552025-04-30T06:44:04.864ZDEBGtest_utilization (store): query key key = _mgs._tcp
75562025-04-30T06:44:04.865ZDEBGtest_utilization (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
75572025-04-30T06:44:04.865ZDEBGtest_utilization (dns): dns response additional_records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal"), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] kind = dns peer_addr = [::1]:58750 query = LowerQuery { name: LowerName(Name("_mgs._tcp.control-plane.oxide.internal.")), original: Query { name: Name("_mgs._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN } } records = [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 48759, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal") })) }] req_id = 0d1ec17b-ceb6-482e-9865-2697af6e480a
75582025-04-30T06:44:04.865ZDEBGtest_utilization (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgs._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 48759, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }], valid_until: Instant { tv_sec: 4309, tv_nsec: 664443074 } })
75592025-04-30T06:44:04.865ZTRACtest_utilization (DnsResolver): lookup_service_targets: looking up SRV target
75602025-04-30T06:44:04.865ZTRACtest_utilization (dns): buffer buffer = Length: 108 (0x6c) bytes\n0000: 93 78 01 00 00 01 00 00 00 00 00 01 2d 64 65 6e .x..........-den\n0010: 64 72 69 74 65 2d 62 36 64 36 35 33 34 31 2d 31 drite-b6d65341-1\n0020: 36 37 63 2d 34 31 64 66 2d 39 62 35 63 2d 34 31 67c-41df-9b5c-41\n0030: 63 64 65 64 39 39 63 32 32 39 04 68 6f 73 74 0d cded99c229.host.\n0040: 63 6f 6e 74 72 6f 6c 2d 70 6c 61 6e 65 05 6f 78 control-plane.ox\n0050: 69 64 65 08 69 6e 74 65 72 6e 61 6c 00 00 1c 00 ide.internal....\n0060: 01 00 00 29 04 d0 00 00 00 00 00 00 ...)........ kind = dns peer_addr = [::1]:63185 req_id = 3d935887-2f57-4300-accf-3b03d9fd0a8f
75612025-04-30T06:44:04.865ZDEBGtest_utilization (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 37752,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n query_type: AAAA,\n query_class: IN,\n },\n },\n original: [\n 45,\n 100,\n 101,\n 110,\n 100,\n 114,\n 105,\n 116,\n 101,\n 45,\n 98,\n 54,\n 100,\n 54,\n 53,\n 51,\n 52,\n 49,\n 45,\n 49,\n 54,\n 55,\n 99,\n 45,\n 52,\n 49,\n 100,\n 102,\n 45,\n 57,\n 98,\n 53,\n 99,\n 45,\n 52,\n 49,\n 99,\n 100,\n 101,\n 100,\n 57,\n 57,\n 99,\n 50,\n 50,\n 57,\n 4,\n 104,\n 111,\n 115,\n 116,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 28,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:63185 req_id = 3d935887-2f57-4300-accf-3b03d9fd0a8f
75622025-04-30T06:44:04.866ZDEBGtest_utilization (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
75632025-04-30T06:44:04.866ZDEBGtest_utilization (dns): dns response additional_records = [] kind = dns peer_addr = [::1]:63185 query = LowerQuery { name: LowerName(Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.")), original: Query { name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), query_type: AAAA, query_class: IN } } records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] req_id = 3d935887-2f57-4300-accf-3b03d9fd0a8f
75642025-04-30T06:44:04.930ZTRACtest_utilization (DnsResolver): lookup_all_socket_v6 srv dns_name = _clickhouse-admin-keeper._tcp.control-plane.oxide.internal
75652025-04-30T06:44:04.931ZTRACtest_utilization (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory
75662025-04-30T06:44:04.931ZTRACtest_utilization (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
75672025-04-30T06:44:04.931ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
75682025-04-30T06:44:04.931ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
75692025-04-30T06:44:04.931ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
75702025-04-30T06:44:04.932ZTRACtest_utilization (dns): buffer buffer = Length: 87 (0x57) bytes\n0000: ca aa 01 00 00 01 00 00 00 00 00 01 18 5f 63 6c ............._cl\n0010: 69 63 6b 68 6f 75 73 65 2d 61 64 6d 69 6e 2d 6b ickhouse-admin-k\n0020: 65 65 70 65 72 04 5f 74 63 70 0d 63 6f 6e 74 72 eeper._tcp.contr\n0030: 6f 6c 2d 70 6c 61 6e 65 05 6f 78 69 64 65 08 69 ol-plane.oxide.i\n0040: 6e 74 65 72 6e 61 6c 00 00 21 00 01 00 00 29 04 nternal..!....).\n0050: d0 00 00 00 00 00 00 ....... kind = dns peer_addr = [::1]:59502 req_id = bd9705be-f73d-494b-a69b-137c5335bde8
75712025-04-30T06:44:04.932ZDEBGtest_utilization (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 51882,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 24,\n 95,\n 99,\n 108,\n 105,\n 99,\n 107,\n 104,\n 111,\n 117,\n 115,\n 101,\n 45,\n 97,\n 100,\n 109,\n 105,\n 110,\n 45,\n 107,\n 101,\n 101,\n 112,\n 101,\n 114,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:59502 req_id = bd9705be-f73d-494b-a69b-137c5335bde8
75722025-04-30T06:44:04.932ZDEBGtest_utilization (store): query key key = _clickhouse-admin-keeper._tcp
75732025-04-30T06:44:04.932ZERROtest_utilization (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 51882, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 1, }, query: WireQuery { query: LowerQuery { name: LowerName( Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), ), original: Query { name: Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN, }, }, original: [ 24, 95, 99, 108, 105, 99, 107, 104, 111, 117, 115, 101, 45, 97, 100, 109, 105, 110, 45, 107, 101, 101, 112, 101, 114, 4, 95, 116, 99, 112, 13, 99, 111, 110, 116, 114, 111, 108, 45, 112, 108, 97, 110, 101, 5, 111, 120, 105, 100, 101, 8, 105, 110, 116, 101, 114, 110, 97, 108, 0, 0, 33, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: Some( Edns { rcode_high: 0, version: 0, dnssec_ok: false, max_payload: 1232, options: OPT { options: {}, }, }, ), } NXDOMAIN: no records found for name: "_clickhouse-admin-keeper._tcp.control-plane.oxide.internal." kind = dns peer_addr = [::1]:59502 req_id = bd9705be-f73d-494b-a69b-137c5335bde8
75742025-04-30T06:44:04.933ZDEBGtest_utilization (BackgroundTasks): begin collection background_task = inventory_collection
75752025-04-30T06:44:04.933ZDEBGtest_utilization (BackgroundTasks): begin collection from MGS background_task = inventory_collection mgs_url = http://[::1]:48759
75762025-04-30T06:44:04.933ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/ignition
75772025-04-30T06:44:04.934ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
75782025-04-30T06:44:04.947ZDEBGtest_utilization (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory result = Ok(())
75792025-04-30T06:44:04.947ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
75802025-04-30T06:44:04.947ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
75812025-04-30T06:44:04.949ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
75822025-04-30T06:44:04.951ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/ignition", status: 200, headers: {"content-type": "application/json", "x-request-id": "537b13dd-6244-4c37-85a9-9fe6c95a8174", "content-length": "835", "date": "Wed, 30 Apr 2025 06:44:04 GMT"} })
75832025-04-30T06:44:04.953ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0
75842025-04-30T06:44:04.953ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
75852025-04-30T06:44:04.953ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
75862025-04-30T06:44:04.954ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
75872025-04-30T06:44:04.957ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "b9b32ee6-69e8-4938-af81-d582e5bd75d4", "content-length": "734", "date": "Wed, 30 Apr 2025 06:44:04 GMT"} })
75882025-04-30T06:44:04.957ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0/component/sp/caboose?firmware_slot=0
75892025-04-30T06:44:04.964ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "681f82ec-4b34-4058-9a0f-4c19bb8271ef", "content-length": "111", "date": "Wed, 30 Apr 2025 06:44:04 GMT"} })
75902025-04-30T06:44:04.964ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0/component/sp/caboose?firmware_slot=1
75912025-04-30T06:44:04.967ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "c27aa095-11ed-4773-9e30-1b2e12752cd4", "content-length": "111", "date": "Wed, 30 Apr 2025 06:44:04 GMT"} })
75922025-04-30T06:44:04.967ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0/component/rot/caboose?firmware_slot=0
75932025-04-30T06:44:04.970ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "b5b915a4-45d3-4f54-b43d-2eef9278eb08", "content-length": "170", "date": "Wed, 30 Apr 2025 06:44:04 GMT"} })
75942025-04-30T06:44:04.970ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0/component/rot/caboose?firmware_slot=1
75952025-04-30T06:44:04.972ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "c26064f4-8746-4162-b993-9954c1b2bfdf", "content-length": "170", "date": "Wed, 30 Apr 2025 06:44:04 GMT"} })
75962025-04-30T06:44:04.973ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0/component/stage0/caboose?firmware_slot=0
75972025-04-30T06:44:04.984ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "9c2a6e48-63c2-4033-b95e-f3e4816e4e72", "content-length": "179", "date": "Wed, 30 Apr 2025 06:44:04 GMT"} })
75982025-04-30T06:44:04.984ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0/component/stage0/caboose?firmware_slot=1
75992025-04-30T06:44:04.988ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "8d349b5d-0847-48e2-afbe-863829c36fc1", "content-length": "179", "date": "Wed, 30 Apr 2025 06:44:04 GMT"} })
76002025-04-30T06:44:04.988ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0/component/rot/cmpa
76012025-04-30T06:44:04.989ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "8c7459b6-8a76-487d-8c25-243a822c6c19", "content-length": "702", "date": "Wed, 30 Apr 2025 06:44:04 GMT"} })
76022025-04-30T06:44:04.989ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0/component/rot/cfpa
76032025-04-30T06:44:04.992ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "42910e8b-cb70-4c45-ba1a-edc4e91cf818", "content-length": "727", "date": "Wed, 30 Apr 2025 06:44:04 GMT"} })
76042025-04-30T06:44:04.993ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0/component/rot/cfpa
76052025-04-30T06:44:04.993ZDEBGtest_utilization: attempt to fetch latest inventory collection result = Ok("found")
76062025-04-30T06:44:04.994ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "505b33bf-70be-4c4c-ac75-7d021b194e61", "content-length": "729", "date": "Wed, 30 Apr 2025 06:44:04 GMT"} })
76072025-04-30T06:44:04.994ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0/component/rot/cfpa
76082025-04-30T06:44:04.995ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "bfb61364-0bb5-4c49-a5f9-08bd57514d9b", "content-length": "728", "date": "Wed, 30 Apr 2025 06:44:04 GMT"} })
76092025-04-30T06:44:04.996ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1
76102025-04-30T06:44:04.996ZDEBGtest_utilization (collection-task): sent timer-based collection request to the collection task collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
76112025-04-30T06:44:04.996ZDEBGtest_utilization (collection-task): collecting from producer collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
76122025-04-30T06:44:04.998ZTRACtest_utilization (collection-task): sent collection request to producer collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
76132025-04-30T06:44:04.999ZDEBGtest_utilization (collection-task): collected results from producer collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 n_results = 1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
76142025-04-30T06:44:04.999ZTRACtest_utilization (collection-task): forwarded results to main collection loop collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
76152025-04-30T06:44:04.999ZTRACtest_utilization (collection-task): top of inner collection loop, waiting for next request collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
76162025-04-30T06:44:05.001ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1", status: 200, headers: {"content-type": "application/json", "x-request-id": "c9ba911a-b571-406a-97e0-38e8667c18a9", "content-length": "734", "date": "Wed, 30 Apr 2025 06:44:04 GMT"} })
76172025-04-30T06:44:05.001ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1/component/sp/caboose?firmware_slot=0
76182025-04-30T06:44:05.004ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "6bbae32a-2822-462f-977a-8dab6ae40c04", "content-length": "111", "date": "Wed, 30 Apr 2025 06:44:04 GMT"} })
76192025-04-30T06:44:05.004ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1/component/sp/caboose?firmware_slot=1
76202025-04-30T06:44:05.007ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "e3bf194c-ba1a-474d-a028-ce1bb5aed6ef", "content-length": "111", "date": "Wed, 30 Apr 2025 06:44:04 GMT"} })
76212025-04-30T06:44:05.007ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1/component/rot/caboose?firmware_slot=0
76222025-04-30T06:44:05.010ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "897576b1-3b0e-4d4e-9720-44fa380e17f9", "content-length": "170", "date": "Wed, 30 Apr 2025 06:44:04 GMT"} })
76232025-04-30T06:44:05.010ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1/component/rot/caboose?firmware_slot=1
76242025-04-30T06:44:05.012ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "36f40c10-f6f5-463b-8a8f-b24d35b8fb1b", "content-length": "170", "date": "Wed, 30 Apr 2025 06:44:04 GMT"} })
76252025-04-30T06:44:05.012ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1/component/stage0/caboose?firmware_slot=0
76262025-04-30T06:44:05.015ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "3a887f2c-6e4a-4f7b-9afe-ab55bedcc396", "content-length": "179", "date": "Wed, 30 Apr 2025 06:44:04 GMT"} })
76272025-04-30T06:44:05.015ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1/component/stage0/caboose?firmware_slot=1
76282025-04-30T06:44:05.017ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "f22155b0-30b5-4dc4-9bc5-34ca3c704a21", "content-length": "179", "date": "Wed, 30 Apr 2025 06:44:04 GMT"} })
76292025-04-30T06:44:05.017ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1/component/rot/cmpa
76302025-04-30T06:44:05.018ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "c39753bb-7013-4ab0-b997-673d09d503a5", "content-length": "702", "date": "Wed, 30 Apr 2025 06:44:04 GMT"} })
76312025-04-30T06:44:05.018ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1/component/rot/cfpa
76322025-04-30T06:44:05.019ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "3e1b942e-c05f-41f3-90fd-0b0c9850f040", "content-length": "727", "date": "Wed, 30 Apr 2025 06:44:04 GMT"} })
76332025-04-30T06:44:05.019ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1/component/rot/cfpa
76342025-04-30T06:44:05.020ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "28657c71-b5ef-47b0-8543-3924122104ce", "content-length": "729", "date": "Wed, 30 Apr 2025 06:44:04 GMT"} })
76352025-04-30T06:44:05.020ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1/component/rot/cfpa
76362025-04-30T06:44:05.021ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "fd2990b7-3c17-4a78-a35b-d494ddedcb42", "content-length": "728", "date": "Wed, 30 Apr 2025 06:44:04 GMT"} })
76372025-04-30T06:44:05.021ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0
76382025-04-30T06:44:05.023ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "35104394-42aa-45c4-83ac-b9481f153057", "content-length": "723", "date": "Wed, 30 Apr 2025 06:44:04 GMT"} })
76392025-04-30T06:44:05.023ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0/component/sp/caboose?firmware_slot=0
76402025-04-30T06:44:05.025ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "d5308d59-5699-454c-85d0-bb56d8b5ddc7", "content-length": "109", "date": "Wed, 30 Apr 2025 06:44:04 GMT"} })
76412025-04-30T06:44:05.026ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0/component/sp/caboose?firmware_slot=1
76422025-04-30T06:44:05.028ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "d601012d-6124-451a-be3b-436084b4424c", "content-length": "109", "date": "Wed, 30 Apr 2025 06:44:04 GMT"} })
76432025-04-30T06:44:05.028ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0/component/rot/caboose?firmware_slot=0
76442025-04-30T06:44:05.030ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "c6a4620a-02a6-4995-aa2a-31009ffcf941", "content-length": "169", "date": "Wed, 30 Apr 2025 06:44:04 GMT"} })
76452025-04-30T06:44:05.030ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0/component/rot/caboose?firmware_slot=1
76462025-04-30T06:44:05.036ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "72800f9c-c881-4323-bdd3-4b2bdefa2339", "content-length": "169", "date": "Wed, 30 Apr 2025 06:44:04 GMT"} })
76472025-04-30T06:44:05.037ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0/component/stage0/caboose?firmware_slot=0
76482025-04-30T06:44:05.039ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "41d62278-5cf5-442b-ad37-a003ac3fbbde", "content-length": "178", "date": "Wed, 30 Apr 2025 06:44:04 GMT"} })
76492025-04-30T06:44:05.039ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0/component/stage0/caboose?firmware_slot=1
76502025-04-30T06:44:05.043ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "53d87328-73df-4808-aa5b-aff1d19e883f", "content-length": "178", "date": "Wed, 30 Apr 2025 06:44:04 GMT"} })
76512025-04-30T06:44:05.043ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0/component/rot/cmpa
76522025-04-30T06:44:05.044ZTRACtest_utilization (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory
76532025-04-30T06:44:05.044ZTRACtest_utilization (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
76542025-04-30T06:44:05.044ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
76552025-04-30T06:44:05.045ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
76562025-04-30T06:44:05.046ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
76572025-04-30T06:44:05.047ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "770367d8-b06e-4bbd-a57c-5a5359d626be", "content-length": "702", "date": "Wed, 30 Apr 2025 06:44:04 GMT"} })
76582025-04-30T06:44:05.047ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0/component/rot/cfpa
76592025-04-30T06:44:05.048ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "9d42c547-539e-49bb-943c-917f275e29bc", "content-length": "727", "date": "Wed, 30 Apr 2025 06:44:04 GMT"} })
76602025-04-30T06:44:05.049ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0/component/rot/cfpa
76612025-04-30T06:44:05.049ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
76622025-04-30T06:44:05.060ZDEBGtest_utilization (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory result = Ok(())
76632025-04-30T06:44:05.060ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
76642025-04-30T06:44:05.060ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
76652025-04-30T06:44:05.061ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
76662025-04-30T06:44:05.063ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "c9aaada0-ff4d-454b-9d36-5fa01390bc1c", "content-length": "729", "date": "Wed, 30 Apr 2025 06:44:04 GMT"} })
76672025-04-30T06:44:05.064ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0/component/rot/cfpa
76682025-04-30T06:44:05.067ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
76692025-04-30T06:44:05.068ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
76702025-04-30T06:44:05.068ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
76712025-04-30T06:44:05.081ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "69449ffc-2368-4300-9038-a47b492e80e8", "content-length": "728", "date": "Wed, 30 Apr 2025 06:44:04 GMT"} })
76722025-04-30T06:44:05.081ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1
76732025-04-30T06:44:05.081ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1", status: 200, headers: {"content-type": "application/json", "x-request-id": "c1e76eee-b7bb-4b2e-978d-761c7f315f6a", "content-length": "723", "date": "Wed, 30 Apr 2025 06:44:04 GMT"} })
76742025-04-30T06:44:05.081ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1/component/sp/caboose?firmware_slot=0
76752025-04-30T06:44:05.084ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "ae4277ca-4cc7-4dbc-9fd4-e5ade9134c83", "content-length": "109", "date": "Wed, 30 Apr 2025 06:44:04 GMT"} })
76762025-04-30T06:44:05.084ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1/component/sp/caboose?firmware_slot=1
76772025-04-30T06:44:05.087ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "46d08b4a-2f71-4ad8-9d4b-6b19a3ba4b2e", "content-length": "109", "date": "Wed, 30 Apr 2025 06:44:04 GMT"} })
76782025-04-30T06:44:05.087ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1/component/rot/caboose?firmware_slot=0
76792025-04-30T06:44:05.091ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "fa141ba7-faaa-4604-aa47-ff5c7a4875f4", "content-length": "169", "date": "Wed, 30 Apr 2025 06:44:04 GMT"} })
76802025-04-30T06:44:05.091ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1/component/rot/caboose?firmware_slot=1
76812025-04-30T06:44:05.099ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "bca5ae6e-a341-4c74-9f51-42b79470f359", "content-length": "169", "date": "Wed, 30 Apr 2025 06:44:04 GMT"} })
76822025-04-30T06:44:05.099ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1/component/stage0/caboose?firmware_slot=0
76832025-04-30T06:44:05.101ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "757d5c0e-20e9-4741-ae71-1bc579ce8911", "content-length": "178", "date": "Wed, 30 Apr 2025 06:44:04 GMT"} })
76842025-04-30T06:44:05.101ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1/component/stage0/caboose?firmware_slot=1
76852025-04-30T06:44:05.110ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "90659c91-c0aa-423f-93f1-809994cae2d6", "content-length": "178", "date": "Wed, 30 Apr 2025 06:44:04 GMT"} })
76862025-04-30T06:44:05.111ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1/component/rot/cmpa
76872025-04-30T06:44:05.131ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "140c61e0-d20c-4f5a-a630-170737cde23f", "content-length": "702", "date": "Wed, 30 Apr 2025 06:44:04 GMT"} })
76882025-04-30T06:44:05.131ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1/component/rot/cfpa
76892025-04-30T06:44:05.131ZDEBGtest_utilization: attempt to fetch latest inventory collection result = Ok("found")
76902025-04-30T06:44:05.135ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "bc18dd2f-754a-4b78-bc3e-b437558ada36", "content-length": "727", "date": "Wed, 30 Apr 2025 06:44:04 GMT"} })
76912025-04-30T06:44:05.136ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1/component/rot/cfpa
76922025-04-30T06:44:05.137ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "175b6f29-87cb-40d0-a090-071dc02cf6f5", "content-length": "729", "date": "Wed, 30 Apr 2025 06:44:04 GMT"} })
76932025-04-30T06:44:05.138ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1/component/rot/cfpa
76942025-04-30T06:44:05.138ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "412b0018-c4c9-4d4e-8db0-05132c13de0d", "content-length": "728", "date": "Wed, 30 Apr 2025 06:44:04 GMT"} })
76952025-04-30T06:44:05.139ZTRACtest_utilization (BackgroundTasks): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet
76962025-04-30T06:44:05.139ZTRACtest_utilization (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
76972025-04-30T06:44:05.139ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
76982025-04-30T06:44:05.139ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
76992025-04-30T06:44:05.140ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
77002025-04-30T06:44:05.155ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
77012025-04-30T06:44:05.160ZDEBGtest_utilization (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet result = Ok(())
77022025-04-30T06:44:05.160ZTRACtest_utilization (BackgroundTasks): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet
77032025-04-30T06:44:05.160ZTRACtest_utilization (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
77042025-04-30T06:44:05.160ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
77052025-04-30T06:44:05.160ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
77062025-04-30T06:44:05.161ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
77072025-04-30T06:44:05.164ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
77082025-04-30T06:44:05.169ZDEBGtest_utilization (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet result = Ok(())
77092025-04-30T06:44:05.169ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
77102025-04-30T06:44:05.169ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
77112025-04-30T06:44:05.170ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
77122025-04-30T06:44:05.255ZDEBGtest_utilization (BackgroundTasks): begin collection from Sled Agent background_task = inventory_collection sled_agent_url = http://[::1]:39365
77132025-04-30T06:44:05.255ZDEBGtest_utilization (BackgroundTasks): client request SledAgent = http://[::1]:39365 background_task = inventory_collection body = None method = GET uri = http://[::1]:39365/inventory
77142025-04-30T06:44:05.255ZTRACtest_utilization (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory
77152025-04-30T06:44:05.255ZTRACtest_utilization (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
77162025-04-30T06:44:05.255ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
77172025-04-30T06:44:05.255ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
77182025-04-30T06:44:05.256ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
77192025-04-30T06:44:05.256ZINFOtest_utilization (dropshot): accepted connection local_addr = [::1]:39365 remote_addr = [::1]:63913 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
77202025-04-30T06:44:05.259ZTRACtest_utilization (dropshot): incoming request local_addr = [::1]:39365 method = GET remote_addr = [::1]:63913 req_id = a9abdaf3-795d-4286-becc-32051464d42d sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /inventory
77212025-04-30T06:44:05.260ZINFOtest_utilization (dropshot): request completed latency_us = 596 local_addr = [::1]:39365 method = GET remote_addr = [::1]:63913 req_id = a9abdaf3-795d-4286-becc-32051464d42d response_code = 200 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /inventory
77222025-04-30T06:44:05.261ZDEBGtest_utilization (BackgroundTasks): client response SledAgent = http://[::1]:39365 background_task = inventory_collection result = Ok(Response { url: "http://[::1]:39365/inventory", status: 200, headers: {"content-type": "application/json", "x-request-id": "a9abdaf3-795d-4286-becc-32051464d42d", "content-length": "3141", "date": "Wed, 30 Apr 2025 06:44:04 GMT"} })
77232025-04-30T06:44:05.267ZDEBGtest_utilization (BackgroundTasks): begin collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
77242025-04-30T06:44:05.267ZDEBGtest_utilization (BackgroundTasks): end collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
77252025-04-30T06:44:05.267ZDEBGtest_utilization (BackgroundTasks): finished collection background_task = inventory_collection
77262025-04-30T06:44:05.270ZTRACtest_utilization (BackgroundTasks): authorize begin action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory
77272025-04-30T06:44:05.270ZTRACtest_utilization (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
77282025-04-30T06:44:05.270ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
77292025-04-30T06:44:05.270ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
77302025-04-30T06:44:05.271ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
77312025-04-30T06:44:05.271ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
77322025-04-30T06:44:05.282ZDEBGtest_utilization (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory result = Ok(())
77332025-04-30T06:44:05.283ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
77342025-04-30T06:44:05.283ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
77352025-04-30T06:44:05.283ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
77362025-04-30T06:44:05.285ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
77372025-04-30T06:44:05.302ZDEBGtest_utilization (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory result = Ok(())
77382025-04-30T06:44:05.303ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
77392025-04-30T06:44:05.303ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
77402025-04-30T06:44:05.303ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
77412025-04-30T06:44:05.306ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
77422025-04-30T06:44:05.306ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
77432025-04-30T06:44:05.307ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
77442025-04-30T06:44:05.360ZDEBGtest_utilization (results-sink): inserting 1 samples into database collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
77452025-04-30T06:44:05.360ZDEBGtest_utilization (results-sink-cluster): inserting 1 samples into database collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
77462025-04-30T06:44:05.360ZDEBGtest_utilization (clickhouse-client): unrolling 1 total samples collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 4d1485f0-110e-4161-9a33-c4610d68be5d
77472025-04-30T06:44:05.371ZTRACtest_utilization (clickhouse-client): inserting data collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 4d1485f0-110e-4161-9a33-c4610d68be5d n_columns = 4 n_rows = 2 sql = INSERT INTO oximeter.fields_string FORMAT Native
77482025-04-30T06:44:05.376ZDEBGtest_utilization (clickhouse-client): inserted rows into table collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 4d1485f0-110e-4161-9a33-c4610d68be5d n_rows = 2 table_name = fields_string
77492025-04-30T06:44:05.376ZTRACtest_utilization (clickhouse-client): inserting data collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 4d1485f0-110e-4161-9a33-c4610d68be5d n_columns = 4 n_rows = 1 sql = INSERT INTO oximeter.measurements_i64 FORMAT Native
77502025-04-30T06:44:05.383ZDEBGtest_utilization (clickhouse-client): inserted rows into table collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 4d1485f0-110e-4161-9a33-c4610d68be5d n_rows = 1 table_name = measurements_i64
77512025-04-30T06:44:05.383ZTRACtest_utilization (results-sink): successfully inserted samples collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
77522025-04-30T06:44:05.383ZWARNtest_utilization (datastore): Non-retryable transaction failure err = DatabaseError(UniqueViolation, "duplicate key value violates unique constraint \\"inv_caboose_pkey\\"") transaction = inventory_insert_collection
77532025-04-30T06:44:05.384ZWARNtest_utilization (BackgroundTasks): inventory collection failed background_task = inventory_collection error = failed to collect inventory: saving inventory to database: Internal Error: unexpected database error: duplicate key value violates unique constraint "inv_caboose_pkey"
77542025-04-30T06:44:05.384ZDEBGtest_utilization (BackgroundTasks): activation complete background_task = inventory_collection elapsed = 532.424732ms iteration = 11 status = {"error":"failed to collect inventory: saving inventory to database: Internal Error: unexpected database error: duplicate key value violates unique constraint \\"inv_caboose_pkey\\""}
77552025-04-30T06:44:05.384ZDEBGtest_utilization (BackgroundTasks): activating background_task = inventory_collection iteration = 12 reason = Signaled
77562025-04-30T06:44:05.384ZTRACtest_utilization (BackgroundTasks): authorize begin action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory
77572025-04-30T06:44:05.384ZTRACtest_utilization (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
77582025-04-30T06:44:05.384ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
77592025-04-30T06:44:05.384ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
77602025-04-30T06:44:05.385ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
77612025-04-30T06:44:05.391ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
77622025-04-30T06:44:05.398ZDEBGtest_utilization (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory result = Ok(())
77632025-04-30T06:44:05.398ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
77642025-04-30T06:44:05.398ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
77652025-04-30T06:44:05.399ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
77662025-04-30T06:44:05.400ZDEBGtest_utilization: attempt to fetch latest inventory collection result = Ok("found")
77672025-04-30T06:44:05.405ZDEBGtest_utilization (BackgroundTasks): inventory_prune_one: nothing eligible for removal (too few) background_task = inventory_collection candidates = [(13d8b8d9-6c6b-47a6-9190-952544edb75b, 0)]
77682025-04-30T06:44:05.405ZTRACtest_utilization (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal
77692025-04-30T06:44:05.406ZTRACtest_utilization (dns): buffer buffer = Length: 67 (0x43) bytes\n0000: 11 40 01 00 00 01 00 00 00 00 00 01 04 5f 6d 67 .@..........._mg\n0010: 73 04 5f 74 63 70 0d 63 6f 6e 74 72 6f 6c 2d 70 s._tcp.control-p\n0020: 6c 61 6e 65 05 6f 78 69 64 65 08 69 6e 74 65 72 lane.oxide.inter\n0030: 6e 61 6c 00 00 21 00 01 00 00 29 04 d0 00 00 00 nal..!....).....\n0040: 00 00 00 ... kind = dns peer_addr = [::1]:53069 req_id = 9c51da1c-d2a4-4748-8043-d07f6ef3fdd4
77702025-04-30T06:44:05.406ZDEBGtest_utilization (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 4416,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_mgs._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_mgs._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 4,\n 95,\n 109,\n 103,\n 115,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:53069 req_id = 9c51da1c-d2a4-4748-8043-d07f6ef3fdd4
77712025-04-30T06:44:05.407ZDEBGtest_utilization (store): query key key = _mgs._tcp
77722025-04-30T06:44:05.407ZDEBGtest_utilization (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
77732025-04-30T06:44:05.407ZDEBGtest_utilization (dns): dns response additional_records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal"), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] kind = dns peer_addr = [::1]:53069 query = LowerQuery { name: LowerName(Name("_mgs._tcp.control-plane.oxide.internal.")), original: Query { name: Name("_mgs._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN } } records = [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 48759, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal") })) }] req_id = 9c51da1c-d2a4-4748-8043-d07f6ef3fdd4
77742025-04-30T06:44:05.407ZDEBGtest_utilization (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgs._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 48759, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }], valid_until: Instant { tv_sec: 4310, tv_nsec: 206895455 } })
77752025-04-30T06:44:05.408ZTRACtest_utilization (DnsResolver): lookup_service_targets: looking up SRV target
77762025-04-30T06:44:05.408ZTRACtest_utilization (dns): buffer buffer = Length: 108 (0x6c) bytes\n0000: 49 b6 01 00 00 01 00 00 00 00 00 01 2d 64 65 6e I...........-den\n0010: 64 72 69 74 65 2d 62 36 64 36 35 33 34 31 2d 31 drite-b6d65341-1\n0020: 36 37 63 2d 34 31 64 66 2d 39 62 35 63 2d 34 31 67c-41df-9b5c-41\n0030: 63 64 65 64 39 39 63 32 32 39 04 68 6f 73 74 0d cded99c229.host.\n0040: 63 6f 6e 74 72 6f 6c 2d 70 6c 61 6e 65 05 6f 78 control-plane.ox\n0050: 69 64 65 08 69 6e 74 65 72 6e 61 6c 00 00 1c 00 ide.internal....\n0060: 01 00 00 29 04 d0 00 00 00 00 00 00 ...)........ kind = dns peer_addr = [::1]:56796 req_id = 32a21435-3d85-4c14-ae6c-e24d8d89d323
77772025-04-30T06:44:05.408ZDEBGtest_utilization (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 18870,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n query_type: AAAA,\n query_class: IN,\n },\n },\n original: [\n 45,\n 100,\n 101,\n 110,\n 100,\n 114,\n 105,\n 116,\n 101,\n 45,\n 98,\n 54,\n 100,\n 54,\n 53,\n 51,\n 52,\n 49,\n 45,\n 49,\n 54,\n 55,\n 99,\n 45,\n 52,\n 49,\n 100,\n 102,\n 45,\n 57,\n 98,\n 53,\n 99,\n 45,\n 52,\n 49,\n 99,\n 100,\n 101,\n 100,\n 57,\n 57,\n 99,\n 50,\n 50,\n 57,\n 4,\n 104,\n 111,\n 115,\n 116,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 28,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:56796 req_id = 32a21435-3d85-4c14-ae6c-e24d8d89d323
77782025-04-30T06:44:05.408ZDEBGtest_utilization (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
77792025-04-30T06:44:05.408ZDEBGtest_utilization (dns): dns response additional_records = [] kind = dns peer_addr = [::1]:56796 query = LowerQuery { name: LowerName(Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.")), original: Query { name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), query_type: AAAA, query_class: IN } } records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] req_id = 32a21435-3d85-4c14-ae6c-e24d8d89d323
77802025-04-30T06:44:05.469ZTRACtest_utilization (DnsResolver): lookup_all_socket_v6 srv dns_name = _clickhouse-admin-keeper._tcp.control-plane.oxide.internal
77812025-04-30T06:44:05.469ZTRACtest_utilization (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory
77822025-04-30T06:44:05.470ZTRACtest_utilization (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
77832025-04-30T06:44:05.470ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
77842025-04-30T06:44:05.470ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
77852025-04-30T06:44:05.474ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
77862025-04-30T06:44:05.474ZWARNtest_utilization (results-sink-cluster): failed to insert some results into metric DB collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 error = Failed to check out connection to database: No backends found for this service file = oximeter/collector/src/results_sink.rs:92
77872025-04-30T06:44:05.474ZTRACtest_utilization (dns): buffer buffer = Length: 87 (0x57) bytes\n0000: d9 c9 01 00 00 01 00 00 00 00 00 01 18 5f 63 6c ............._cl\n0010: 69 63 6b 68 6f 75 73 65 2d 61 64 6d 69 6e 2d 6b ickhouse-admin-k\n0020: 65 65 70 65 72 04 5f 74 63 70 0d 63 6f 6e 74 72 eeper._tcp.contr\n0030: 6f 6c 2d 70 6c 61 6e 65 05 6f 78 69 64 65 08 69 ol-plane.oxide.i\n0040: 6e 74 65 72 6e 61 6c 00 00 21 00 01 00 00 29 04 nternal..!....).\n0050: d0 00 00 00 00 00 00 ....... kind = dns peer_addr = [::1]:60288 req_id = 07b0b4a0-38d1-41c0-87fa-335619051449
77882025-04-30T06:44:05.474ZDEBGtest_utilization (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 55753,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 24,\n 95,\n 99,\n 108,\n 105,\n 99,\n 107,\n 104,\n 111,\n 117,\n 115,\n 101,\n 45,\n 97,\n 100,\n 109,\n 105,\n 110,\n 45,\n 107,\n 101,\n 101,\n 112,\n 101,\n 114,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:60288 req_id = 07b0b4a0-38d1-41c0-87fa-335619051449
77892025-04-30T06:44:05.474ZDEBGtest_utilization (store): query key key = _clickhouse-admin-keeper._tcp
77902025-04-30T06:44:05.475ZERROtest_utilization (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 55753, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 1, }, query: WireQuery { query: LowerQuery { name: LowerName( Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), ), original: Query { name: Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN, }, }, original: [ 24, 95, 99, 108, 105, 99, 107, 104, 111, 117, 115, 101, 45, 97, 100, 109, 105, 110, 45, 107, 101, 101, 112, 101, 114, 4, 95, 116, 99, 112, 13, 99, 111, 110, 116, 114, 111, 108, 45, 112, 108, 97, 110, 101, 5, 111, 120, 105, 100, 101, 8, 105, 110, 116, 101, 114, 110, 97, 108, 0, 0, 33, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: Some( Edns { rcode_high: 0, version: 0, dnssec_ok: false, max_payload: 1232, options: OPT { options: {}, }, }, ), } NXDOMAIN: no records found for name: "_clickhouse-admin-keeper._tcp.control-plane.oxide.internal." kind = dns peer_addr = [::1]:60288 req_id = 07b0b4a0-38d1-41c0-87fa-335619051449
77912025-04-30T06:44:05.475ZDEBGtest_utilization (BackgroundTasks): begin collection background_task = inventory_collection
77922025-04-30T06:44:05.475ZDEBGtest_utilization (BackgroundTasks): begin collection from MGS background_task = inventory_collection mgs_url = http://[::1]:48759
77932025-04-30T06:44:05.475ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/ignition
77942025-04-30T06:44:05.477ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/ignition", status: 200, headers: {"content-type": "application/json", "x-request-id": "4018c86b-2515-4cd8-bade-ab4b1190a2f8", "content-length": "835", "date": "Wed, 30 Apr 2025 06:44:05 GMT"} })
77952025-04-30T06:44:05.478ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0
77962025-04-30T06:44:05.478ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
77972025-04-30T06:44:05.490ZDEBGtest_utilization (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory result = Ok(())
77982025-04-30T06:44:05.490ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
77992025-04-30T06:44:05.490ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
78002025-04-30T06:44:05.490ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
78012025-04-30T06:44:05.494ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
78022025-04-30T06:44:05.494ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
78032025-04-30T06:44:05.494ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
78042025-04-30T06:44:05.497ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "8db6deb3-b541-4e85-8411-c6aaae9a4a72", "content-length": "734", "date": "Wed, 30 Apr 2025 06:44:05 GMT"} })
78052025-04-30T06:44:05.497ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0/component/sp/caboose?firmware_slot=0
78062025-04-30T06:44:05.500ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "f37948e9-f005-43db-b9d2-d2b3bbeca7f1", "content-length": "111", "date": "Wed, 30 Apr 2025 06:44:05 GMT"} })
78072025-04-30T06:44:05.500ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0/component/sp/caboose?firmware_slot=1
78082025-04-30T06:44:05.504ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "a96b5d8d-e098-4810-9240-b6f92f00380b", "content-length": "111", "date": "Wed, 30 Apr 2025 06:44:05 GMT"} })
78092025-04-30T06:44:05.505ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0/component/rot/caboose?firmware_slot=0
78102025-04-30T06:44:05.511ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "f0b87acf-08e7-49f7-9864-91765bc7fbcb", "content-length": "170", "date": "Wed, 30 Apr 2025 06:44:05 GMT"} })
78112025-04-30T06:44:05.511ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0/component/rot/caboose?firmware_slot=1
78122025-04-30T06:44:05.515ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "f9bb5088-4522-43f4-a24d-29cdba680261", "content-length": "170", "date": "Wed, 30 Apr 2025 06:44:05 GMT"} })
78132025-04-30T06:44:05.517ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0/component/stage0/caboose?firmware_slot=0
78142025-04-30T06:44:05.520ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "3046630d-74c2-420c-a1d7-11ad474eb4b8", "content-length": "179", "date": "Wed, 30 Apr 2025 06:44:05 GMT"} })
78152025-04-30T06:44:05.520ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0/component/stage0/caboose?firmware_slot=1
78162025-04-30T06:44:05.525ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "cfaaf1d6-1f0b-4fc9-9535-d320dd835c68", "content-length": "179", "date": "Wed, 30 Apr 2025 06:44:05 GMT"} })
78172025-04-30T06:44:05.525ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0/component/rot/cmpa
78182025-04-30T06:44:05.528ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "ae18356e-ce00-4990-a063-7654d48ccd8a", "content-length": "702", "date": "Wed, 30 Apr 2025 06:44:05 GMT"} })
78192025-04-30T06:44:05.528ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0/component/rot/cfpa
78202025-04-30T06:44:05.530ZDEBGtest_utilization: attempt to fetch latest inventory collection result = Ok("found")
78212025-04-30T06:44:05.531ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "91d22ad6-3766-4262-8c54-a957ff361097", "content-length": "727", "date": "Wed, 30 Apr 2025 06:44:05 GMT"} })
78222025-04-30T06:44:05.531ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0/component/rot/cfpa
78232025-04-30T06:44:05.533ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "e7b8bfdd-f278-4f0b-b8bc-46aa068f0ad5", "content-length": "729", "date": "Wed, 30 Apr 2025 06:44:05 GMT"} })
78242025-04-30T06:44:05.533ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0/component/rot/cfpa
78252025-04-30T06:44:05.534ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "e61428e8-bd20-4484-9228-876f901836b4", "content-length": "728", "date": "Wed, 30 Apr 2025 06:44:05 GMT"} })
78262025-04-30T06:44:05.534ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1
78272025-04-30T06:44:05.539ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1", status: 200, headers: {"content-type": "application/json", "x-request-id": "e091e2ba-af30-4135-9ef2-d66d87e85e5d", "content-length": "734", "date": "Wed, 30 Apr 2025 06:44:05 GMT"} })
78282025-04-30T06:44:05.539ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1/component/sp/caboose?firmware_slot=0
78292025-04-30T06:44:05.548ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "e2161384-8fd8-41f4-bd8e-07f68eff98cf", "content-length": "111", "date": "Wed, 30 Apr 2025 06:44:05 GMT"} })
78302025-04-30T06:44:05.548ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1/component/sp/caboose?firmware_slot=1
78312025-04-30T06:44:05.558ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "8d5329f1-d142-4341-9837-e8f7abfcc4a5", "content-length": "111", "date": "Wed, 30 Apr 2025 06:44:05 GMT"} })
78322025-04-30T06:44:05.558ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1/component/rot/caboose?firmware_slot=0
78332025-04-30T06:44:05.561ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "2634c11a-a03e-472b-9cab-2885baec1cd1", "content-length": "170", "date": "Wed, 30 Apr 2025 06:44:05 GMT"} })
78342025-04-30T06:44:05.561ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1/component/rot/caboose?firmware_slot=1
78352025-04-30T06:44:05.563ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "88909e44-60a1-4c82-a6cc-7a21e35da801", "content-length": "170", "date": "Wed, 30 Apr 2025 06:44:05 GMT"} })
78362025-04-30T06:44:05.563ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1/component/stage0/caboose?firmware_slot=0
78372025-04-30T06:44:05.566ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "c5f6cc60-8360-49d3-a7b2-fcb7fb211b3b", "content-length": "179", "date": "Wed, 30 Apr 2025 06:44:05 GMT"} })
78382025-04-30T06:44:05.566ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1/component/stage0/caboose?firmware_slot=1
78392025-04-30T06:44:05.588ZTRACtest_utilization (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory
78402025-04-30T06:44:05.589ZTRACtest_utilization (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
78412025-04-30T06:44:05.589ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
78422025-04-30T06:44:05.589ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
78432025-04-30T06:44:05.592ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
78442025-04-30T06:44:05.604ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
78452025-04-30T06:44:05.613ZDEBGtest_utilization (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory result = Ok(())
78462025-04-30T06:44:05.613ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
78472025-04-30T06:44:05.613ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
78482025-04-30T06:44:05.614ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
78492025-04-30T06:44:05.616ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "112a3610-d7c5-4326-a93b-a4b2e1c5938e", "content-length": "179", "date": "Wed, 30 Apr 2025 06:44:05 GMT"} })
78502025-04-30T06:44:05.616ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1/component/rot/cmpa
78512025-04-30T06:44:05.620ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "167eb5fa-0ae9-4616-a922-ea14e707a14f", "content-length": "702", "date": "Wed, 30 Apr 2025 06:44:05 GMT"} })
78522025-04-30T06:44:05.620ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1/component/rot/cfpa
78532025-04-30T06:44:05.621ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
78542025-04-30T06:44:05.621ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
78552025-04-30T06:44:05.622ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
78562025-04-30T06:44:05.623ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "2a12fbba-d3e5-405b-b1db-6a330b070087", "content-length": "727", "date": "Wed, 30 Apr 2025 06:44:05 GMT"} })
78572025-04-30T06:44:05.623ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1/component/rot/cfpa
78582025-04-30T06:44:05.625ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "a57895e6-47de-430b-bd59-5289fd4e522d", "content-length": "729", "date": "Wed, 30 Apr 2025 06:44:05 GMT"} })
78592025-04-30T06:44:05.625ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1/component/rot/cfpa
78602025-04-30T06:44:05.626ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "bbc67384-842b-4708-ac1f-d52332b430d7", "content-length": "728", "date": "Wed, 30 Apr 2025 06:44:05 GMT"} })
78612025-04-30T06:44:05.626ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0
78622025-04-30T06:44:05.628ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "5af46b81-3470-49c9-b664-758b22146659", "content-length": "723", "date": "Wed, 30 Apr 2025 06:44:05 GMT"} })
78632025-04-30T06:44:05.629ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0/component/sp/caboose?firmware_slot=0
78642025-04-30T06:44:05.633ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "9f6d65f2-cb98-498a-b383-499fccccb651", "content-length": "109", "date": "Wed, 30 Apr 2025 06:44:05 GMT"} })
78652025-04-30T06:44:05.633ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0/component/sp/caboose?firmware_slot=1
78662025-04-30T06:44:05.638ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "bcd3685f-e8db-448e-bb65-08c95d6a3ab8", "content-length": "109", "date": "Wed, 30 Apr 2025 06:44:05 GMT"} })
78672025-04-30T06:44:05.638ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0/component/rot/caboose?firmware_slot=0
78682025-04-30T06:44:05.648ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "cf1a9194-8cba-4dc2-a857-0e09610a58ec", "content-length": "169", "date": "Wed, 30 Apr 2025 06:44:05 GMT"} })
78692025-04-30T06:44:05.648ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0/component/rot/caboose?firmware_slot=1
78702025-04-30T06:44:05.652ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "b22859cf-5955-4fe5-9259-161ba7b55dec", "content-length": "169", "date": "Wed, 30 Apr 2025 06:44:05 GMT"} })
78712025-04-30T06:44:05.652ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0/component/stage0/caboose?firmware_slot=0
78722025-04-30T06:44:05.655ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "7980c91d-19de-4cb8-95e6-3e3c5052a68a", "content-length": "178", "date": "Wed, 30 Apr 2025 06:44:05 GMT"} })
78732025-04-30T06:44:05.655ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0/component/stage0/caboose?firmware_slot=1
78742025-04-30T06:44:05.658ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "f7fd51b3-ed00-400a-8c58-a9e4cdecb71c", "content-length": "178", "date": "Wed, 30 Apr 2025 06:44:05 GMT"} })
78752025-04-30T06:44:05.658ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0/component/rot/cmpa
78762025-04-30T06:44:05.659ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "191056b5-0395-419e-aa77-acb2b712fd05", "content-length": "702", "date": "Wed, 30 Apr 2025 06:44:05 GMT"} })
78772025-04-30T06:44:05.659ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0/component/rot/cfpa
78782025-04-30T06:44:05.663ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "89685a76-9664-4bd0-9e60-6d5494b6d0a7", "content-length": "727", "date": "Wed, 30 Apr 2025 06:44:05 GMT"} })
78792025-04-30T06:44:05.663ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0/component/rot/cfpa
78802025-04-30T06:44:05.664ZDEBGtest_utilization: attempt to fetch latest inventory collection result = Ok("found")
78812025-04-30T06:44:05.675ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "c62584f0-68a2-4b45-a5af-8a1f7bba1687", "content-length": "729", "date": "Wed, 30 Apr 2025 06:44:05 GMT"} })
78822025-04-30T06:44:05.676ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0/component/rot/cfpa
78832025-04-30T06:44:05.679ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "2e1e105b-a770-4d18-abd9-4e6622c6f29b", "content-length": "728", "date": "Wed, 30 Apr 2025 06:44:05 GMT"} })
78842025-04-30T06:44:05.679ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1
78852025-04-30T06:44:05.681ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1", status: 200, headers: {"content-type": "application/json", "x-request-id": "18f90d00-5798-466c-a9ab-8af13f484b5a", "content-length": "723", "date": "Wed, 30 Apr 2025 06:44:05 GMT"} })
78862025-04-30T06:44:05.689ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1/component/sp/caboose?firmware_slot=0
78872025-04-30T06:44:05.692ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "e6b88578-c8f2-4eb3-b85b-72f4d3bf414d", "content-length": "109", "date": "Wed, 30 Apr 2025 06:44:05 GMT"} })
78882025-04-30T06:44:05.693ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1/component/sp/caboose?firmware_slot=1
78892025-04-30T06:44:05.698ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "95d087e4-6f0e-42a3-be7c-ceb6d8a28c1b", "content-length": "109", "date": "Wed, 30 Apr 2025 06:44:05 GMT"} })
78902025-04-30T06:44:05.698ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1/component/rot/caboose?firmware_slot=0
78912025-04-30T06:44:05.703ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "4343246e-22b5-47ea-a336-5aa37ab2897a", "content-length": "169", "date": "Wed, 30 Apr 2025 06:44:05 GMT"} })
78922025-04-30T06:44:05.703ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1/component/rot/caboose?firmware_slot=1
78932025-04-30T06:44:05.705ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "e4c590d3-58ac-4764-a883-8162dbf46845", "content-length": "169", "date": "Wed, 30 Apr 2025 06:44:05 GMT"} })
78942025-04-30T06:44:05.705ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1/component/stage0/caboose?firmware_slot=0
78952025-04-30T06:44:05.709ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "a80737cc-1e05-4736-a3b3-088bf91c75a6", "content-length": "178", "date": "Wed, 30 Apr 2025 06:44:05 GMT"} })
78962025-04-30T06:44:05.709ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1/component/stage0/caboose?firmware_slot=1
78972025-04-30T06:44:05.712ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "2a0b26c3-69c4-43e7-bd2b-b336555f4965", "content-length": "178", "date": "Wed, 30 Apr 2025 06:44:05 GMT"} })
78982025-04-30T06:44:05.712ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1/component/rot/cmpa
78992025-04-30T06:44:05.713ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "81f276f4-69d0-4fed-b67b-f95ad5cea562", "content-length": "702", "date": "Wed, 30 Apr 2025 06:44:05 GMT"} })
79002025-04-30T06:44:05.714ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1/component/rot/cfpa
79012025-04-30T06:44:05.717ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "b94338b1-561b-4150-8b65-6c3eb636d87a", "content-length": "727", "date": "Wed, 30 Apr 2025 06:44:05 GMT"} })
79022025-04-30T06:44:05.717ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1/component/rot/cfpa
79032025-04-30T06:44:05.718ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "3706a45a-a8d8-4427-a048-4cabc17fc97f", "content-length": "729", "date": "Wed, 30 Apr 2025 06:44:05 GMT"} })
79042025-04-30T06:44:05.718ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1/component/rot/cfpa
79052025-04-30T06:44:05.719ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "2f6c6591-7188-44cf-8260-4b8b36ae2422", "content-length": "728", "date": "Wed, 30 Apr 2025 06:44:05 GMT"} })
79062025-04-30T06:44:05.720ZTRACtest_utilization (BackgroundTasks): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet
79072025-04-30T06:44:05.720ZTRACtest_utilization (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
79082025-04-30T06:44:05.720ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
79092025-04-30T06:44:05.720ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
79102025-04-30T06:44:05.720ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
79112025-04-30T06:44:05.721ZTRACtest_utilization (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory
79122025-04-30T06:44:05.721ZTRACtest_utilization (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
79132025-04-30T06:44:05.721ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
79142025-04-30T06:44:05.721ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
79152025-04-30T06:44:05.722ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
79162025-04-30T06:44:05.724ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
79172025-04-30T06:44:05.731ZDEBGtest_utilization (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet result = Ok(())
79182025-04-30T06:44:05.733ZTRACtest_utilization (BackgroundTasks): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet
79192025-04-30T06:44:05.733ZTRACtest_utilization (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
79202025-04-30T06:44:05.733ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
79212025-04-30T06:44:05.733ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
79222025-04-30T06:44:05.735ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
79232025-04-30T06:44:05.737ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
79242025-04-30T06:44:05.751ZDEBGtest_utilization (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory result = Ok(())
79252025-04-30T06:44:05.751ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
79262025-04-30T06:44:05.751ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
79272025-04-30T06:44:05.752ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
79282025-04-30T06:44:05.757ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
79292025-04-30T06:44:05.763ZDEBGtest_utilization (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet result = Ok(())
79302025-04-30T06:44:05.763ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
79312025-04-30T06:44:05.763ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
79322025-04-30T06:44:05.765ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
79332025-04-30T06:44:05.766ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
79342025-04-30T06:44:05.766ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
79352025-04-30T06:44:05.768ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
79362025-04-30T06:44:05.847ZDEBGtest_utilization (BackgroundTasks): begin collection from Sled Agent background_task = inventory_collection sled_agent_url = http://[::1]:39365
79372025-04-30T06:44:05.848ZDEBGtest_utilization (BackgroundTasks): client request SledAgent = http://[::1]:39365 background_task = inventory_collection body = None method = GET uri = http://[::1]:39365/inventory
79382025-04-30T06:44:05.851ZINFOtest_utilization (dropshot): accepted connection local_addr = [::1]:39365 remote_addr = [::1]:48192 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
79392025-04-30T06:44:05.855ZTRACtest_utilization (dropshot): incoming request local_addr = [::1]:39365 method = GET remote_addr = [::1]:48192 req_id = 759e1b86-fe5a-4ab2-8874-cfc29312047b sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /inventory
79402025-04-30T06:44:05.856ZINFOtest_utilization (dropshot): request completed latency_us = 1039 local_addr = [::1]:39365 method = GET remote_addr = [::1]:48192 req_id = 759e1b86-fe5a-4ab2-8874-cfc29312047b response_code = 200 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /inventory
79412025-04-30T06:44:05.856ZDEBGtest_utilization (BackgroundTasks): client response SledAgent = http://[::1]:39365 background_task = inventory_collection result = Ok(Response { url: "http://[::1]:39365/inventory", status: 200, headers: {"content-type": "application/json", "x-request-id": "759e1b86-fe5a-4ab2-8874-cfc29312047b", "content-length": "3141", "date": "Wed, 30 Apr 2025 06:44:05 GMT"} })
79422025-04-30T06:44:05.860ZDEBGtest_utilization (BackgroundTasks): begin collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
79432025-04-30T06:44:05.860ZDEBGtest_utilization (BackgroundTasks): end collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
79442025-04-30T06:44:05.860ZDEBGtest_utilization (BackgroundTasks): finished collection background_task = inventory_collection
79452025-04-30T06:44:05.863ZTRACtest_utilization (BackgroundTasks): authorize begin action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory
79462025-04-30T06:44:05.863ZTRACtest_utilization (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
79472025-04-30T06:44:05.863ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
79482025-04-30T06:44:05.863ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
79492025-04-30T06:44:05.864ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
79502025-04-30T06:44:05.868ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
79512025-04-30T06:44:05.875ZDEBGtest_utilization (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory result = Ok(())
79522025-04-30T06:44:05.876ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
79532025-04-30T06:44:05.876ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
79542025-04-30T06:44:05.876ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
79552025-04-30T06:44:05.909ZDEBGtest_utilization: attempt to fetch latest inventory collection result = Ok("found")
79562025-04-30T06:44:05.916ZWARNtest_utilization (datastore): Non-retryable transaction failure err = DatabaseError(UniqueViolation, "duplicate key value violates unique constraint \\"inv_caboose_pkey\\"") transaction = inventory_insert_collection
79572025-04-30T06:44:05.917ZWARNtest_utilization (BackgroundTasks): inventory collection failed background_task = inventory_collection error = failed to collect inventory: saving inventory to database: Internal Error: unexpected database error: duplicate key value violates unique constraint "inv_caboose_pkey"
79582025-04-30T06:44:05.917ZDEBGtest_utilization (BackgroundTasks): activation complete background_task = inventory_collection elapsed = 532.94603ms iteration = 12 status = {"error":"failed to collect inventory: saving inventory to database: Internal Error: unexpected database error: duplicate key value violates unique constraint \\"inv_caboose_pkey\\""}
79592025-04-30T06:44:05.917ZDEBGtest_utilization (BackgroundTasks): activating background_task = inventory_collection iteration = 13 reason = Signaled
79602025-04-30T06:44:05.917ZTRACtest_utilization (BackgroundTasks): authorize begin action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory
79612025-04-30T06:44:05.917ZTRACtest_utilization (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
79622025-04-30T06:44:05.917ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
79632025-04-30T06:44:05.917ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
79642025-04-30T06:44:05.918ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
79652025-04-30T06:44:05.922ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
79662025-04-30T06:44:05.934ZDEBGtest_utilization (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory result = Ok(())
79672025-04-30T06:44:05.935ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
79682025-04-30T06:44:05.935ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
79692025-04-30T06:44:05.935ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
79702025-04-30T06:44:05.940ZDEBGtest_utilization (BackgroundTasks): inventory_prune_one: nothing eligible for removal (too few) background_task = inventory_collection candidates = [(13d8b8d9-6c6b-47a6-9190-952544edb75b, 0)]
79712025-04-30T06:44:05.940ZTRACtest_utilization (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal
79722025-04-30T06:44:05.941ZTRACtest_utilization (dns): buffer buffer = Length: 67 (0x43) bytes\n0000: 90 b3 01 00 00 01 00 00 00 00 00 01 04 5f 6d 67 ............._mg\n0010: 73 04 5f 74 63 70 0d 63 6f 6e 74 72 6f 6c 2d 70 s._tcp.control-p\n0020: 6c 61 6e 65 05 6f 78 69 64 65 08 69 6e 74 65 72 lane.oxide.inter\n0030: 6e 61 6c 00 00 21 00 01 00 00 29 04 d0 00 00 00 nal..!....).....\n0040: 00 00 00 ... kind = dns peer_addr = [::1]:56986 req_id = e3535c78-4d57-4ff6-9f91-096fcb421ccd
79732025-04-30T06:44:05.941ZDEBGtest_utilization (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 37043,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_mgs._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_mgs._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 4,\n 95,\n 109,\n 103,\n 115,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:56986 req_id = e3535c78-4d57-4ff6-9f91-096fcb421ccd
79742025-04-30T06:44:05.941ZDEBGtest_utilization (store): query key key = _mgs._tcp
79752025-04-30T06:44:05.941ZDEBGtest_utilization (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
79762025-04-30T06:44:05.941ZDEBGtest_utilization (dns): dns response additional_records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal"), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] kind = dns peer_addr = [::1]:56986 query = LowerQuery { name: LowerName(Name("_mgs._tcp.control-plane.oxide.internal.")), original: Query { name: Name("_mgs._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN } } records = [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 48759, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal") })) }] req_id = e3535c78-4d57-4ff6-9f91-096fcb421ccd
79772025-04-30T06:44:05.942ZDEBGtest_utilization (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgs._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 48759, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }], valid_until: Instant { tv_sec: 4310, tv_nsec: 741276454 } })
79782025-04-30T06:44:05.942ZTRACtest_utilization (DnsResolver): lookup_service_targets: looking up SRV target
79792025-04-30T06:44:05.942ZTRACtest_utilization (dns): buffer buffer = Length: 108 (0x6c) bytes\n0000: 06 75 01 00 00 01 00 00 00 00 00 01 2d 64 65 6e .u..........-den\n0010: 64 72 69 74 65 2d 62 36 64 36 35 33 34 31 2d 31 drite-b6d65341-1\n0020: 36 37 63 2d 34 31 64 66 2d 39 62 35 63 2d 34 31 67c-41df-9b5c-41\n0030: 63 64 65 64 39 39 63 32 32 39 04 68 6f 73 74 0d cded99c229.host.\n0040: 63 6f 6e 74 72 6f 6c 2d 70 6c 61 6e 65 05 6f 78 control-plane.ox\n0050: 69 64 65 08 69 6e 74 65 72 6e 61 6c 00 00 1c 00 ide.internal....\n0060: 01 00 00 29 04 d0 00 00 00 00 00 00 ...)........ kind = dns peer_addr = [::1]:52322 req_id = 22483304-bb49-4d4f-8334-42b362d7d63f
79802025-04-30T06:44:05.943ZDEBGtest_utilization (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 1653,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n query_type: AAAA,\n query_class: IN,\n },\n },\n original: [\n 45,\n 100,\n 101,\n 110,\n 100,\n 114,\n 105,\n 116,\n 101,\n 45,\n 98,\n 54,\n 100,\n 54,\n 53,\n 51,\n 52,\n 49,\n 45,\n 49,\n 54,\n 55,\n 99,\n 45,\n 52,\n 49,\n 100,\n 102,\n 45,\n 57,\n 98,\n 53,\n 99,\n 45,\n 52,\n 49,\n 99,\n 100,\n 101,\n 100,\n 57,\n 57,\n 99,\n 50,\n 50,\n 57,\n 4,\n 104,\n 111,\n 115,\n 116,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 28,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:52322 req_id = 22483304-bb49-4d4f-8334-42b362d7d63f
79812025-04-30T06:44:05.943ZDEBGtest_utilization (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
79822025-04-30T06:44:05.943ZDEBGtest_utilization (dns): dns response additional_records = [] kind = dns peer_addr = [::1]:52322 query = LowerQuery { name: LowerName(Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.")), original: Query { name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), query_type: AAAA, query_class: IN } } records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] req_id = 22483304-bb49-4d4f-8334-42b362d7d63f
79832025-04-30T06:44:06.011ZTRACtest_utilization (DnsResolver): lookup_all_socket_v6 srv dns_name = _clickhouse-admin-keeper._tcp.control-plane.oxide.internal
79842025-04-30T06:44:06.011ZTRACtest_utilization (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory
79852025-04-30T06:44:06.011ZTRACtest_utilization (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
79862025-04-30T06:44:06.011ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
79872025-04-30T06:44:06.011ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
79882025-04-30T06:44:06.012ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
79892025-04-30T06:44:06.012ZINFOtest_utilization (oximeter-agent): refreshing list of producers from Nexus collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 file = oximeter/collector/src/agent.rs:562
79902025-04-30T06:44:06.012ZDEBGtest_utilization (collection-task): sent timer-based collection request to the collection task collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
79912025-04-30T06:44:06.012ZDEBGtest_utilization (collection-task): collecting from producer collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
79922025-04-30T06:44:06.013ZDEBGtest_utilization (oximeter): client request body = None method = GET uri = http://127.0.0.1:32956/metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?sort_by=id_ascending
79932025-04-30T06:44:06.013ZTRACtest_utilization (dropshot_internal): incoming request local_addr = 127.0.0.1:32956 method = GET remote_addr = 127.0.0.1:55797 req_id = 4c92ab2c-3b18-4bca-8a16-dd88ac402f76 uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?sort_by=id_ascending
79942025-04-30T06:44:06.014ZTRACtest_utilization (dns): buffer buffer = Length: 87 (0x57) bytes\n0000: 9c d1 01 00 00 01 00 00 00 00 00 01 18 5f 63 6c ............._cl\n0010: 69 63 6b 68 6f 75 73 65 2d 61 64 6d 69 6e 2d 6b ickhouse-admin-k\n0020: 65 65 70 65 72 04 5f 74 63 70 0d 63 6f 6e 74 72 eeper._tcp.contr\n0030: 6f 6c 2d 70 6c 61 6e 65 05 6f 78 69 64 65 08 69 ol-plane.oxide.i\n0040: 6e 74 65 72 6e 61 6c 00 00 21 00 01 00 00 29 04 nternal..!....).\n0050: d0 00 00 00 00 00 00 ....... kind = dns peer_addr = [::1]:58315 req_id = 70d0cdd1-c5ce-48bd-afd4-5b8bf6e42877
79952025-04-30T06:44:06.014ZDEBGtest_utilization (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 40145,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 24,\n 95,\n 99,\n 108,\n 105,\n 99,\n 107,\n 104,\n 111,\n 117,\n 115,\n 101,\n 45,\n 97,\n 100,\n 109,\n 105,\n 110,\n 45,\n 107,\n 101,\n 101,\n 112,\n 101,\n 114,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:58315 req_id = 70d0cdd1-c5ce-48bd-afd4-5b8bf6e42877
79962025-04-30T06:44:06.014ZDEBGtest_utilization (store): query key key = _clickhouse-admin-keeper._tcp
79972025-04-30T06:44:06.014ZERROtest_utilization (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 40145, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 1, }, query: WireQuery { query: LowerQuery { name: LowerName( Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), ), original: Query { name: Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN, }, }, original: [ 24, 95, 99, 108, 105, 99, 107, 104, 111, 117, 115, 101, 45, 97, 100, 109, 105, 110, 45, 107, 101, 101, 112, 101, 114, 4, 95, 116, 99, 112, 13, 99, 111, 110, 116, 114, 111, 108, 45, 112, 108, 97, 110, 101, 5, 111, 120, 105, 100, 101, 8, 105, 110, 116, 101, 114, 110, 97, 108, 0, 0, 33, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: Some( Edns { rcode_high: 0, version: 0, dnssec_ok: false, max_payload: 1232, options: OPT { options: {}, }, }, ), } NXDOMAIN: no records found for name: "_clickhouse-admin-keeper._tcp.control-plane.oxide.internal." kind = dns peer_addr = [::1]:58315 req_id = 70d0cdd1-c5ce-48bd-afd4-5b8bf6e42877
79982025-04-30T06:44:06.015ZTRACtest_utilization (dropshot_internal): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = 127.0.0.1:32956 method = GET remote_addr = 127.0.0.1:55797 req_id = 4c92ab2c-3b18-4bca-8a16-dd88ac402f76 resource = Database uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?sort_by=id_ascending
79992025-04-30T06:44:06.015ZDEBGtest_utilization (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = 127.0.0.1:32956 method = GET remote_addr = 127.0.0.1:55797 req_id = 4c92ab2c-3b18-4bca-8a16-dd88ac402f76 roles = RoleSet { roles: {} } uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?sort_by=id_ascending
80002025-04-30T06:44:06.015ZDEBGtest_utilization (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = 127.0.0.1:32956 method = GET remote_addr = 127.0.0.1:55797 req_id = 4c92ab2c-3b18-4bca-8a16-dd88ac402f76 resource = Database result = Ok(()) uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?sort_by=id_ascending
80012025-04-30T06:44:06.019ZDEBGtest_utilization (BackgroundTasks): begin collection background_task = inventory_collection
80022025-04-30T06:44:06.019ZDEBGtest_utilization (BackgroundTasks): begin collection from MGS background_task = inventory_collection mgs_url = http://[::1]:48759
80032025-04-30T06:44:06.019ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/ignition
80042025-04-30T06:44:06.019ZTRACtest_utilization (collection-task): sent collection request to producer collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
80052025-04-30T06:44:06.020ZDEBGtest_utilization (collection-task): collected results from producer collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 n_results = 1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
80062025-04-30T06:44:06.020ZTRACtest_utilization (collection-task): forwarded results to main collection loop collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
80072025-04-30T06:44:06.020ZTRACtest_utilization (collection-task): top of inner collection loop, waiting for next request collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
80082025-04-30T06:44:06.021ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
80092025-04-30T06:44:06.026ZDEBGtest_utilization (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory result = Ok(())
80102025-04-30T06:44:06.027ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
80112025-04-30T06:44:06.027ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
80122025-04-30T06:44:06.027ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
80132025-04-30T06:44:06.028ZINFOtest_utilization (dropshot_internal): request completed latency_us = 14563 local_addr = 127.0.0.1:32956 method = GET remote_addr = 127.0.0.1:55797 req_id = 4c92ab2c-3b18-4bca-8a16-dd88ac402f76 response_code = 200 uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?sort_by=id_ascending
80142025-04-30T06:44:06.032ZDEBGtest_utilization (oximeter): client response result = Ok(Response { url: "http://127.0.0.1:32956/metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?sort_by=id_ascending", status: 200, headers: {"content-type": "application/json", "x-request-id": "4c92ab2c-3b18-4bca-8a16-dd88ac402f76", "content-length": "405", "date": "Wed, 30 Apr 2025 06:44:05 GMT"} })
80152025-04-30T06:44:06.032ZDEBGtest_utilization (oximeter): client request body = None method = GET uri = http://127.0.0.1:32956/metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJhYzU1ZDhkOS1hMjk4LTQ1MzQtODk4Yi1iZWVlMWM5ZTRlNTIifX0%3D
80162025-04-30T06:44:06.032ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
80172025-04-30T06:44:06.032ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
80182025-04-30T06:44:06.034ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
80192025-04-30T06:44:06.034ZTRACtest_utilization (dropshot_internal): incoming request local_addr = 127.0.0.1:32956 method = GET remote_addr = 127.0.0.1:55797 req_id = 50bc2f58-dc89-4ab6-9159-ccecc180939e uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJhYzU1ZDhkOS1hMjk4LTQ1MzQtODk4Yi1iZWVlMWM5ZTRlNTIifX0%3D
80202025-04-30T06:44:06.035ZTRACtest_utilization (dropshot_internal): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = 127.0.0.1:32956 method = GET remote_addr = 127.0.0.1:55797 req_id = 50bc2f58-dc89-4ab6-9159-ccecc180939e resource = Database uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJhYzU1ZDhkOS1hMjk4LTQ1MzQtODk4Yi1iZWVlMWM5ZTRlNTIifX0%3D
80212025-04-30T06:44:06.035ZDEBGtest_utilization (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = 127.0.0.1:32956 method = GET remote_addr = 127.0.0.1:55797 req_id = 50bc2f58-dc89-4ab6-9159-ccecc180939e roles = RoleSet { roles: {} } uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJhYzU1ZDhkOS1hMjk4LTQ1MzQtODk4Yi1iZWVlMWM5ZTRlNTIifX0%3D
80222025-04-30T06:44:06.038ZDEBGtest_utilization (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = 127.0.0.1:32956 method = GET remote_addr = 127.0.0.1:55797 req_id = 50bc2f58-dc89-4ab6-9159-ccecc180939e resource = Database result = Ok(()) uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJhYzU1ZDhkOS1hMjk4LTQ1MzQtODk4Yi1iZWVlMWM5ZTRlNTIifX0%3D
80232025-04-30T06:44:06.043ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/ignition", status: 200, headers: {"content-type": "application/json", "x-request-id": "d02df522-1182-4351-98f0-b45276086bc3", "content-length": "835", "date": "Wed, 30 Apr 2025 06:44:05 GMT"} })
80242025-04-30T06:44:06.043ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0
80252025-04-30T06:44:06.045ZINFOtest_utilization (dropshot_internal): request completed latency_us = 11386 local_addr = 127.0.0.1:32956 method = GET remote_addr = 127.0.0.1:55797 req_id = 50bc2f58-dc89-4ab6-9159-ccecc180939e response_code = 200 uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJhYzU1ZDhkOS1hMjk4LTQ1MzQtODk4Yi1iZWVlMWM5ZTRlNTIifX0%3D
80262025-04-30T06:44:06.046ZDEBGtest_utilization (oximeter): client response result = Ok(Response { url: "http://127.0.0.1:32956/metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJhYzU1ZDhkOS1hMjk4LTQ1MzQtODk4Yi1iZWVlMWM5ZTRlNTIifX0%3D", status: 200, headers: {"content-type": "application/json", "x-request-id": "50bc2f58-dc89-4ab6-9159-ccecc180939e", "content-length": "29", "date": "Wed, 30 Apr 2025 06:44:05 GMT"} })
80272025-04-30T06:44:06.048ZTRACtest_utilization (oximeter-agent): ignoring request to update existing metric producer, since the endpoint information is the same as the existing collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
80282025-04-30T06:44:06.048ZTRACtest_utilization (oximeter-agent): ignoring request to update existing metric producer, since the endpoint information is the same as the existing collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = ac55d8d9-a298-4534-898b-beee1c9e4e52
80292025-04-30T06:44:06.048ZINFOtest_utilization (oximeter-agent): refreshed list of producers from Nexus collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 file = oximeter/collector/src/agent.rs:637 n_current_tasks = 2 n_pruned_tasks = 0
80302025-04-30T06:44:06.050ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "b25ad746-d5df-4657-bb85-b548f1b5839b", "content-length": "734", "date": "Wed, 30 Apr 2025 06:44:05 GMT"} })
80312025-04-30T06:44:06.051ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0/component/sp/caboose?firmware_slot=0
80322025-04-30T06:44:06.061ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "03395f80-000e-4df2-a533-7536553f9a9d", "content-length": "111", "date": "Wed, 30 Apr 2025 06:44:05 GMT"} })
80332025-04-30T06:44:06.061ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0/component/sp/caboose?firmware_slot=1
80342025-04-30T06:44:06.096ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "da60cd53-3d69-48d5-975b-de743199bcd8", "content-length": "111", "date": "Wed, 30 Apr 2025 06:44:05 GMT"} })
80352025-04-30T06:44:06.096ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0/component/rot/caboose?firmware_slot=0
80362025-04-30T06:44:06.100ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "d7be3f35-c45d-4610-855b-73ae17897888", "content-length": "170", "date": "Wed, 30 Apr 2025 06:44:05 GMT"} })
80372025-04-30T06:44:06.100ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0/component/rot/caboose?firmware_slot=1
80382025-04-30T06:44:06.107ZDEBGtest_utilization: attempt to fetch latest inventory collection result = Ok("found")
80392025-04-30T06:44:06.109ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "411f91e8-d1e1-41f3-af3a-3323914984c8", "content-length": "170", "date": "Wed, 30 Apr 2025 06:44:05 GMT"} })
80402025-04-30T06:44:06.110ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0/component/stage0/caboose?firmware_slot=0
80412025-04-30T06:44:06.112ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "5ea337ee-b828-4af6-96b8-8acf72757471", "content-length": "179", "date": "Wed, 30 Apr 2025 06:44:05 GMT"} })
80422025-04-30T06:44:06.113ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0/component/stage0/caboose?firmware_slot=1
80432025-04-30T06:44:06.115ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "4a2a4d04-06bc-4d8d-b9ee-f016feb70e65", "content-length": "179", "date": "Wed, 30 Apr 2025 06:44:05 GMT"} })
80442025-04-30T06:44:06.115ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0/component/rot/cmpa
80452025-04-30T06:44:06.116ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "ef5a8046-1e9f-4b7e-8a5f-49ac7f081549", "content-length": "702", "date": "Wed, 30 Apr 2025 06:44:05 GMT"} })
80462025-04-30T06:44:06.116ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0/component/rot/cfpa
80472025-04-30T06:44:06.117ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "dacf7af8-f6eb-48e7-8e28-f20417f1f72a", "content-length": "727", "date": "Wed, 30 Apr 2025 06:44:05 GMT"} })
80482025-04-30T06:44:06.118ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0/component/rot/cfpa
80492025-04-30T06:44:06.119ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "66c517ca-fd1e-41d0-aa3a-00195acad767", "content-length": "729", "date": "Wed, 30 Apr 2025 06:44:05 GMT"} })
80502025-04-30T06:44:06.119ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0/component/rot/cfpa
80512025-04-30T06:44:06.120ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "cf039751-b26f-4396-bfc8-f9608bd1d88f", "content-length": "728", "date": "Wed, 30 Apr 2025 06:44:05 GMT"} })
80522025-04-30T06:44:06.120ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1
80532025-04-30T06:44:06.122ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1", status: 200, headers: {"content-type": "application/json", "x-request-id": "5cf8508d-f133-43b4-972b-b59de022c51e", "content-length": "734", "date": "Wed, 30 Apr 2025 06:44:05 GMT"} })
80542025-04-30T06:44:06.122ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1/component/sp/caboose?firmware_slot=0
80552025-04-30T06:44:06.124ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "2079384e-7a6e-4cf8-ab18-fbd9b10e6267", "content-length": "111", "date": "Wed, 30 Apr 2025 06:44:05 GMT"} })
80562025-04-30T06:44:06.125ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1/component/sp/caboose?firmware_slot=1
80572025-04-30T06:44:06.127ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "647476e6-8d75-4960-8268-75eb6912c91e", "content-length": "111", "date": "Wed, 30 Apr 2025 06:44:05 GMT"} })
80582025-04-30T06:44:06.127ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1/component/rot/caboose?firmware_slot=0
80592025-04-30T06:44:06.130ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "a303cfc9-4663-421c-80e2-9f562b8feee6", "content-length": "170", "date": "Wed, 30 Apr 2025 06:44:05 GMT"} })
80602025-04-30T06:44:06.130ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1/component/rot/caboose?firmware_slot=1
80612025-04-30T06:44:06.133ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "14d3b373-10d0-43a7-94ec-18ec2b7bceb7", "content-length": "170", "date": "Wed, 30 Apr 2025 06:44:05 GMT"} })
80622025-04-30T06:44:06.133ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1/component/stage0/caboose?firmware_slot=0
80632025-04-30T06:44:06.136ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "b33e0aba-323e-455a-bf81-078b8c974524", "content-length": "179", "date": "Wed, 30 Apr 2025 06:44:05 GMT"} })
80642025-04-30T06:44:06.136ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1/component/stage0/caboose?firmware_slot=1
80652025-04-30T06:44:06.138ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "2438c96d-5ffb-4965-b5f5-2a562b416bc7", "content-length": "179", "date": "Wed, 30 Apr 2025 06:44:05 GMT"} })
80662025-04-30T06:44:06.139ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1/component/rot/cmpa
80672025-04-30T06:44:06.139ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "e5957101-40d5-4104-afd2-dcc083b1825d", "content-length": "702", "date": "Wed, 30 Apr 2025 06:44:05 GMT"} })
80682025-04-30T06:44:06.140ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1/component/rot/cfpa
80692025-04-30T06:44:06.141ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "6b3d4dcb-ff0f-49a0-b65b-b1434ab718fe", "content-length": "727", "date": "Wed, 30 Apr 2025 06:44:05 GMT"} })
80702025-04-30T06:44:06.141ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1/component/rot/cfpa
80712025-04-30T06:44:06.142ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "867823f9-85e5-45fd-863e-aab26869795b", "content-length": "729", "date": "Wed, 30 Apr 2025 06:44:05 GMT"} })
80722025-04-30T06:44:06.142ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1/component/rot/cfpa
80732025-04-30T06:44:06.144ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "627b1bf2-e03a-4ed1-aa6c-86ba962f5c41", "content-length": "728", "date": "Wed, 30 Apr 2025 06:44:05 GMT"} })
80742025-04-30T06:44:06.144ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0
80752025-04-30T06:44:06.146ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "bf6d7877-69aa-4eb7-bbee-06dd5382dab4", "content-length": "723", "date": "Wed, 30 Apr 2025 06:44:05 GMT"} })
80762025-04-30T06:44:06.146ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0/component/sp/caboose?firmware_slot=0
80772025-04-30T06:44:06.152ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "e95708b9-1ee9-4fec-9142-ca4596d9e923", "content-length": "109", "date": "Wed, 30 Apr 2025 06:44:05 GMT"} })
80782025-04-30T06:44:06.152ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0/component/sp/caboose?firmware_slot=1
80792025-04-30T06:44:06.162ZTRACtest_utilization (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory
80802025-04-30T06:44:06.162ZTRACtest_utilization (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
80812025-04-30T06:44:06.162ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
80822025-04-30T06:44:06.162ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
80832025-04-30T06:44:06.165ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
80842025-04-30T06:44:06.168ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "041d82c3-9999-48be-b5e1-2366da160ab0", "content-length": "109", "date": "Wed, 30 Apr 2025 06:44:05 GMT"} })
80852025-04-30T06:44:06.168ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0/component/rot/caboose?firmware_slot=0
80862025-04-30T06:44:06.168ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
80872025-04-30T06:44:06.208ZDEBGtest_utilization (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory result = Ok(())
80882025-04-30T06:44:06.208ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
80892025-04-30T06:44:06.208ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
80902025-04-30T06:44:06.208ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
80912025-04-30T06:44:06.212ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "cb23d70a-f704-46b8-9394-d7aada185d58", "content-length": "169", "date": "Wed, 30 Apr 2025 06:44:05 GMT"} })
80922025-04-30T06:44:06.215ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0/component/rot/caboose?firmware_slot=1
80932025-04-30T06:44:06.216ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
80942025-04-30T06:44:06.216ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
80952025-04-30T06:44:06.216ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
80962025-04-30T06:44:06.228ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "3e2559b1-3c20-458c-8b31-c3f918136bb3", "content-length": "169", "date": "Wed, 30 Apr 2025 06:44:05 GMT"} })
80972025-04-30T06:44:06.230ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0/component/stage0/caboose?firmware_slot=0
80982025-04-30T06:44:06.233ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "5eab58f0-a23f-4c3f-bfb4-6c79a6e2f59f", "content-length": "178", "date": "Wed, 30 Apr 2025 06:44:05 GMT"} })
80992025-04-30T06:44:06.234ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0/component/stage0/caboose?firmware_slot=1
81002025-04-30T06:44:06.240ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "4506b50d-531f-4317-b325-77bc057f3320", "content-length": "178", "date": "Wed, 30 Apr 2025 06:44:05 GMT"} })
81012025-04-30T06:44:06.240ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0/component/rot/cmpa
81022025-04-30T06:44:06.242ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "347ce02f-f927-4fec-b1c9-3ab9255f6818", "content-length": "702", "date": "Wed, 30 Apr 2025 06:44:05 GMT"} })
81032025-04-30T06:44:06.242ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0/component/rot/cfpa
81042025-04-30T06:44:06.249ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "5889e4a8-e9e3-4369-be87-7b26a2fe4774", "content-length": "727", "date": "Wed, 30 Apr 2025 06:44:05 GMT"} })
81052025-04-30T06:44:06.249ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0/component/rot/cfpa
81062025-04-30T06:44:06.249ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "37fa872a-6038-4436-b8ce-a1d3959464db", "content-length": "729", "date": "Wed, 30 Apr 2025 06:44:05 GMT"} })
81072025-04-30T06:44:06.249ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0/component/rot/cfpa
81082025-04-30T06:44:06.251ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "1f08d107-5cf7-469b-9dc5-50aa47808d7d", "content-length": "728", "date": "Wed, 30 Apr 2025 06:44:05 GMT"} })
81092025-04-30T06:44:06.251ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1
81102025-04-30T06:44:06.252ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1", status: 200, headers: {"content-type": "application/json", "x-request-id": "3ba38018-3b05-4d2f-9f05-e1829fcc9930", "content-length": "723", "date": "Wed, 30 Apr 2025 06:44:05 GMT"} })
81112025-04-30T06:44:06.252ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1/component/sp/caboose?firmware_slot=0
81122025-04-30T06:44:06.255ZDEBGtest_utilization: attempt to fetch latest inventory collection result = Ok("found")
81132025-04-30T06:44:06.258ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "87c8bd91-455d-4ab3-845e-81ef9788849d", "content-length": "109", "date": "Wed, 30 Apr 2025 06:44:05 GMT"} })
81142025-04-30T06:44:06.258ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1/component/sp/caboose?firmware_slot=1
81152025-04-30T06:44:06.263ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "369eca0e-f2bf-488a-8e08-529a10a27d2c", "content-length": "109", "date": "Wed, 30 Apr 2025 06:44:05 GMT"} })
81162025-04-30T06:44:06.263ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1/component/rot/caboose?firmware_slot=0
81172025-04-30T06:44:06.265ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "5dc2eb7c-23bf-484c-a6f5-0e3c89010827", "content-length": "169", "date": "Wed, 30 Apr 2025 06:44:05 GMT"} })
81182025-04-30T06:44:06.266ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1/component/rot/caboose?firmware_slot=1
81192025-04-30T06:44:06.268ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "d4ed84af-b8cf-4bfd-9b84-312babeb5d55", "content-length": "169", "date": "Wed, 30 Apr 2025 06:44:05 GMT"} })
81202025-04-30T06:44:06.268ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1/component/stage0/caboose?firmware_slot=0
81212025-04-30T06:44:06.272ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "92b2cce6-6951-4d49-a03d-b479cdd841b6", "content-length": "178", "date": "Wed, 30 Apr 2025 06:44:05 GMT"} })
81222025-04-30T06:44:06.272ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1/component/stage0/caboose?firmware_slot=1
81232025-04-30T06:44:06.277ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "63c9328d-21a7-47e0-9a7b-4ca1edc7986e", "content-length": "178", "date": "Wed, 30 Apr 2025 06:44:05 GMT"} })
81242025-04-30T06:44:06.278ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1/component/rot/cmpa
81252025-04-30T06:44:06.278ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "b3be19c1-0440-476e-9658-057b6a34efac", "content-length": "702", "date": "Wed, 30 Apr 2025 06:44:05 GMT"} })
81262025-04-30T06:44:06.280ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1/component/rot/cfpa
81272025-04-30T06:44:06.299ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "05183734-36aa-4383-b781-e556a19e6467", "content-length": "727", "date": "Wed, 30 Apr 2025 06:44:05 GMT"} })
81282025-04-30T06:44:06.299ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1/component/rot/cfpa
81292025-04-30T06:44:06.300ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "4f6d4db2-1e4d-457d-884d-236ecac54802", "content-length": "729", "date": "Wed, 30 Apr 2025 06:44:05 GMT"} })
81302025-04-30T06:44:06.300ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1/component/rot/cfpa
81312025-04-30T06:44:06.301ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "c496b9d9-2898-47a5-ade7-1ed854fb24d1", "content-length": "728", "date": "Wed, 30 Apr 2025 06:44:05 GMT"} })
81322025-04-30T06:44:06.301ZTRACtest_utilization (BackgroundTasks): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet
81332025-04-30T06:44:06.301ZTRACtest_utilization (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
81342025-04-30T06:44:06.301ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
81352025-04-30T06:44:06.301ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
81362025-04-30T06:44:06.302ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
81372025-04-30T06:44:06.311ZTRACtest_utilization (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory
81382025-04-30T06:44:06.313ZTRACtest_utilization (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
81392025-04-30T06:44:06.313ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
81402025-04-30T06:44:06.313ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
81412025-04-30T06:44:06.314ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
81422025-04-30T06:44:06.328ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
81432025-04-30T06:44:06.334ZDEBGtest_utilization (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet result = Ok(())
81442025-04-30T06:44:06.334ZTRACtest_utilization (BackgroundTasks): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet
81452025-04-30T06:44:06.334ZTRACtest_utilization (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
81462025-04-30T06:44:06.334ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
81472025-04-30T06:44:06.334ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
81482025-04-30T06:44:06.335ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
81492025-04-30T06:44:06.337ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
81502025-04-30T06:44:06.344ZDEBGtest_utilization (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory result = Ok(())
81512025-04-30T06:44:06.345ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
81522025-04-30T06:44:06.345ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
81532025-04-30T06:44:06.346ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
81542025-04-30T06:44:06.354ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
81552025-04-30T06:44:06.358ZDEBGtest_utilization (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet result = Ok(())
81562025-04-30T06:44:06.358ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
81572025-04-30T06:44:06.358ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
81582025-04-30T06:44:06.359ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
81592025-04-30T06:44:06.361ZDEBGtest_utilization (results-sink): inserting 1 samples into database collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
81602025-04-30T06:44:06.361ZDEBGtest_utilization (results-sink-cluster): inserting 1 samples into database collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
81612025-04-30T06:44:06.361ZDEBGtest_utilization (clickhouse-client): unrolling 1 total samples collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 4d1485f0-110e-4161-9a33-c4610d68be5d
81622025-04-30T06:44:06.367ZTRACtest_utilization (clickhouse-client): inserting data collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 4d1485f0-110e-4161-9a33-c4610d68be5d n_columns = 4 n_rows = 2 sql = INSERT INTO oximeter.fields_string FORMAT Native
81632025-04-30T06:44:06.425ZDEBGtest_utilization (BackgroundTasks): begin collection from Sled Agent background_task = inventory_collection sled_agent_url = http://[::1]:39365
81642025-04-30T06:44:06.433ZDEBGtest_utilization (BackgroundTasks): client request SledAgent = http://[::1]:39365 background_task = inventory_collection body = None method = GET uri = http://[::1]:39365/inventory
81652025-04-30T06:44:06.434ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
81662025-04-30T06:44:06.434ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
81672025-04-30T06:44:06.435ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
81682025-04-30T06:44:06.436ZINFOtest_utilization (dropshot): accepted connection local_addr = [::1]:39365 remote_addr = [::1]:36461 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
81692025-04-30T06:44:06.441ZDEBGtest_utilization (clickhouse-client): inserted rows into table collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 4d1485f0-110e-4161-9a33-c4610d68be5d n_rows = 2 table_name = fields_string
81702025-04-30T06:44:06.441ZTRACtest_utilization (clickhouse-client): inserting data collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 4d1485f0-110e-4161-9a33-c4610d68be5d n_columns = 4 n_rows = 1 sql = INSERT INTO oximeter.measurements_i64 FORMAT Native
81712025-04-30T06:44:06.442ZTRACtest_utilization (dropshot): incoming request local_addr = [::1]:39365 method = GET remote_addr = [::1]:36461 req_id = f7c69430-9213-41c8-8ce1-3b0b8cc7b25e sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /inventory
81722025-04-30T06:44:06.443ZINFOtest_utilization (dropshot): request completed latency_us = 656 local_addr = [::1]:39365 method = GET remote_addr = [::1]:36461 req_id = f7c69430-9213-41c8-8ce1-3b0b8cc7b25e response_code = 200 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /inventory
81732025-04-30T06:44:06.447ZDEBGtest_utilization (BackgroundTasks): client response SledAgent = http://[::1]:39365 background_task = inventory_collection result = Ok(Response { url: "http://[::1]:39365/inventory", status: 200, headers: {"content-type": "application/json", "x-request-id": "f7c69430-9213-41c8-8ce1-3b0b8cc7b25e", "content-length": "3141", "date": "Wed, 30 Apr 2025 06:44:05 GMT"} })
81742025-04-30T06:44:06.451ZDEBGtest_utilization (BackgroundTasks): begin collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
81752025-04-30T06:44:06.451ZDEBGtest_utilization (BackgroundTasks): end collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
81762025-04-30T06:44:06.451ZDEBGtest_utilization (BackgroundTasks): finished collection background_task = inventory_collection
81772025-04-30T06:44:06.454ZTRACtest_utilization (BackgroundTasks): authorize begin action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory
81782025-04-30T06:44:06.454ZTRACtest_utilization (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
81792025-04-30T06:44:06.454ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
81802025-04-30T06:44:06.454ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
81812025-04-30T06:44:06.455ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
81822025-04-30T06:44:06.456ZDEBGtest_utilization (clickhouse-client): inserted rows into table collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 4d1485f0-110e-4161-9a33-c4610d68be5d n_rows = 1 table_name = measurements_i64
81832025-04-30T06:44:06.456ZTRACtest_utilization (results-sink): successfully inserted samples collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
81842025-04-30T06:44:06.465ZWARNtest_utilization (results-sink-cluster): failed to insert some results into metric DB collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 error = Failed to check out connection to database: No backends found for this service file = oximeter/collector/src/results_sink.rs:92
81852025-04-30T06:44:06.465ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
81862025-04-30T06:44:06.473ZDEBGtest_utilization (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory result = Ok(())
81872025-04-30T06:44:06.473ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
81882025-04-30T06:44:06.474ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
81892025-04-30T06:44:06.474ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
81902025-04-30T06:44:06.500ZDEBGtest_utilization: attempt to fetch latest inventory collection result = Ok("found")
81912025-04-30T06:44:06.512ZWARNtest_utilization (datastore): Non-retryable transaction failure err = DatabaseError(UniqueViolation, "duplicate key value violates unique constraint \\"inv_caboose_pkey\\"") transaction = inventory_insert_collection
81922025-04-30T06:44:06.513ZWARNtest_utilization (BackgroundTasks): inventory collection failed background_task = inventory_collection error = failed to collect inventory: saving inventory to database: Internal Error: unexpected database error: duplicate key value violates unique constraint "inv_caboose_pkey"
81932025-04-30T06:44:06.513ZDEBGtest_utilization (BackgroundTasks): activation complete background_task = inventory_collection elapsed = 595.850083ms iteration = 13 status = {"error":"failed to collect inventory: saving inventory to database: Internal Error: unexpected database error: duplicate key value violates unique constraint \\"inv_caboose_pkey\\""}
81942025-04-30T06:44:06.513ZDEBGtest_utilization (BackgroundTasks): activating background_task = inventory_collection iteration = 14 reason = Signaled
81952025-04-30T06:44:06.513ZTRACtest_utilization (BackgroundTasks): authorize begin action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory
81962025-04-30T06:44:06.513ZTRACtest_utilization (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
81972025-04-30T06:44:06.513ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
81982025-04-30T06:44:06.513ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
81992025-04-30T06:44:06.514ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
82002025-04-30T06:44:06.516ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
82012025-04-30T06:44:06.523ZDEBGtest_utilization (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory result = Ok(())
82022025-04-30T06:44:06.524ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
82032025-04-30T06:44:06.524ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
82042025-04-30T06:44:06.524ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
82052025-04-30T06:44:06.545ZDEBGtest_utilization (BackgroundTasks): inventory_prune_one: nothing eligible for removal (too few) background_task = inventory_collection candidates = [(13d8b8d9-6c6b-47a6-9190-952544edb75b, 0)]
82062025-04-30T06:44:06.545ZTRACtest_utilization (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal
82072025-04-30T06:44:06.546ZTRACtest_utilization (dns): buffer buffer = Length: 67 (0x43) bytes\n0000: 52 df 01 00 00 01 00 00 00 00 00 01 04 5f 6d 67 R............_mg\n0010: 73 04 5f 74 63 70 0d 63 6f 6e 74 72 6f 6c 2d 70 s._tcp.control-p\n0020: 6c 61 6e 65 05 6f 78 69 64 65 08 69 6e 74 65 72 lane.oxide.inter\n0030: 6e 61 6c 00 00 21 00 01 00 00 29 04 d0 00 00 00 nal..!....).....\n0040: 00 00 00 ... kind = dns peer_addr = [::1]:64888 req_id = 886ff466-c10c-46b2-aef7-d7902dfa2f02
82082025-04-30T06:44:06.546ZDEBGtest_utilization (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 21215,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_mgs._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_mgs._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 4,\n 95,\n 109,\n 103,\n 115,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:64888 req_id = 886ff466-c10c-46b2-aef7-d7902dfa2f02
82092025-04-30T06:44:06.546ZDEBGtest_utilization (store): query key key = _mgs._tcp
82102025-04-30T06:44:06.546ZDEBGtest_utilization (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
82112025-04-30T06:44:06.546ZDEBGtest_utilization (dns): dns response additional_records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal"), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] kind = dns peer_addr = [::1]:64888 query = LowerQuery { name: LowerName(Name("_mgs._tcp.control-plane.oxide.internal.")), original: Query { name: Name("_mgs._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN } } records = [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 48759, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal") })) }] req_id = 886ff466-c10c-46b2-aef7-d7902dfa2f02
82122025-04-30T06:44:06.547ZDEBGtest_utilization (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgs._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 48759, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }], valid_until: Instant { tv_sec: 4311, tv_nsec: 345930759 } })
82132025-04-30T06:44:06.547ZTRACtest_utilization (DnsResolver): lookup_service_targets: looking up SRV target
82142025-04-30T06:44:06.547ZTRACtest_utilization (dns): buffer buffer = Length: 108 (0x6c) bytes\n0000: 63 63 01 00 00 01 00 00 00 00 00 01 2d 64 65 6e cc..........-den\n0010: 64 72 69 74 65 2d 62 36 64 36 35 33 34 31 2d 31 drite-b6d65341-1\n0020: 36 37 63 2d 34 31 64 66 2d 39 62 35 63 2d 34 31 67c-41df-9b5c-41\n0030: 63 64 65 64 39 39 63 32 32 39 04 68 6f 73 74 0d cded99c229.host.\n0040: 63 6f 6e 74 72 6f 6c 2d 70 6c 61 6e 65 05 6f 78 control-plane.ox\n0050: 69 64 65 08 69 6e 74 65 72 6e 61 6c 00 00 1c 00 ide.internal....\n0060: 01 00 00 29 04 d0 00 00 00 00 00 00 ...)........ kind = dns peer_addr = [::1]:64717 req_id = e1c708f0-38ef-41e1-a8d3-8cb527ca5755
82152025-04-30T06:44:06.547ZDEBGtest_utilization (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 25443,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n query_type: AAAA,\n query_class: IN,\n },\n },\n original: [\n 45,\n 100,\n 101,\n 110,\n 100,\n 114,\n 105,\n 116,\n 101,\n 45,\n 98,\n 54,\n 100,\n 54,\n 53,\n 51,\n 52,\n 49,\n 45,\n 49,\n 54,\n 55,\n 99,\n 45,\n 52,\n 49,\n 100,\n 102,\n 45,\n 57,\n 98,\n 53,\n 99,\n 45,\n 52,\n 49,\n 99,\n 100,\n 101,\n 100,\n 57,\n 57,\n 99,\n 50,\n 50,\n 57,\n 4,\n 104,\n 111,\n 115,\n 116,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 28,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:64717 req_id = e1c708f0-38ef-41e1-a8d3-8cb527ca5755
82162025-04-30T06:44:06.547ZDEBGtest_utilization (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
82172025-04-30T06:44:06.547ZDEBGtest_utilization (dns): dns response additional_records = [] kind = dns peer_addr = [::1]:64717 query = LowerQuery { name: LowerName(Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.")), original: Query { name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), query_type: AAAA, query_class: IN } } records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] req_id = e1c708f0-38ef-41e1-a8d3-8cb527ca5755
82182025-04-30T06:44:06.602ZTRACtest_utilization (DnsResolver): lookup_all_socket_v6 srv dns_name = _clickhouse-admin-keeper._tcp.control-plane.oxide.internal
82192025-04-30T06:44:06.602ZTRACtest_utilization (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory
82202025-04-30T06:44:06.602ZTRACtest_utilization (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
82212025-04-30T06:44:06.602ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
82222025-04-30T06:44:06.602ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
82232025-04-30T06:44:06.604ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
82242025-04-30T06:44:06.615ZTRACtest_utilization (dns): buffer buffer = Length: 87 (0x57) bytes\n0000: fa b1 01 00 00 01 00 00 00 00 00 01 18 5f 63 6c ............._cl\n0010: 69 63 6b 68 6f 75 73 65 2d 61 64 6d 69 6e 2d 6b ickhouse-admin-k\n0020: 65 65 70 65 72 04 5f 74 63 70 0d 63 6f 6e 74 72 eeper._tcp.contr\n0030: 6f 6c 2d 70 6c 61 6e 65 05 6f 78 69 64 65 08 69 ol-plane.oxide.i\n0040: 6e 74 65 72 6e 61 6c 00 00 21 00 01 00 00 29 04 nternal..!....).\n0050: d0 00 00 00 00 00 00 ....... kind = dns peer_addr = [::1]:57424 req_id = 27256c5f-6f09-46b0-90d7-8139857e85a1
82252025-04-30T06:44:06.615ZDEBGtest_utilization (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 64177,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 24,\n 95,\n 99,\n 108,\n 105,\n 99,\n 107,\n 104,\n 111,\n 117,\n 115,\n 101,\n 45,\n 97,\n 100,\n 109,\n 105,\n 110,\n 45,\n 107,\n 101,\n 101,\n 112,\n 101,\n 114,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:57424 req_id = 27256c5f-6f09-46b0-90d7-8139857e85a1
82262025-04-30T06:44:06.616ZDEBGtest_utilization (store): query key key = _clickhouse-admin-keeper._tcp
82272025-04-30T06:44:06.616ZERROtest_utilization (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 64177, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 1, }, query: WireQuery { query: LowerQuery { name: LowerName( Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), ), original: Query { name: Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN, }, }, original: [ 24, 95, 99, 108, 105, 99, 107, 104, 111, 117, 115, 101, 45, 97, 100, 109, 105, 110, 45, 107, 101, 101, 112, 101, 114, 4, 95, 116, 99, 112, 13, 99, 111, 110, 116, 114, 111, 108, 45, 112, 108, 97, 110, 101, 5, 111, 120, 105, 100, 101, 8, 105, 110, 116, 101, 114, 110, 97, 108, 0, 0, 33, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: Some( Edns { rcode_high: 0, version: 0, dnssec_ok: false, max_payload: 1232, options: OPT { options: {}, }, }, ), } NXDOMAIN: no records found for name: "_clickhouse-admin-keeper._tcp.control-plane.oxide.internal." kind = dns peer_addr = [::1]:57424 req_id = 27256c5f-6f09-46b0-90d7-8139857e85a1
82282025-04-30T06:44:06.622ZDEBGtest_utilization (BackgroundTasks): begin collection background_task = inventory_collection
82292025-04-30T06:44:06.622ZDEBGtest_utilization (BackgroundTasks): begin collection from MGS background_task = inventory_collection mgs_url = http://[::1]:48759
82302025-04-30T06:44:06.622ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/ignition
82312025-04-30T06:44:06.625ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
82322025-04-30T06:44:06.636ZDEBGtest_utilization (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory result = Ok(())
82332025-04-30T06:44:06.636ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
82342025-04-30T06:44:06.636ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
82352025-04-30T06:44:06.637ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
82362025-04-30T06:44:06.647ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
82372025-04-30T06:44:06.647ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
82382025-04-30T06:44:06.648ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
82392025-04-30T06:44:06.655ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/ignition", status: 200, headers: {"content-type": "application/json", "x-request-id": "7a51801b-d27e-47d4-aedc-fefd8bbb0bda", "content-length": "835", "date": "Wed, 30 Apr 2025 06:44:06 GMT"} })
82402025-04-30T06:44:06.656ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0
82412025-04-30T06:44:06.670ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "387c52f4-6766-4d76-8d9d-3c52e823de13", "content-length": "734", "date": "Wed, 30 Apr 2025 06:44:06 GMT"} })
82422025-04-30T06:44:06.671ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0/component/sp/caboose?firmware_slot=0
82432025-04-30T06:44:06.687ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "0f84470f-ab2e-49b4-afbd-09e75ea185f4", "content-length": "111", "date": "Wed, 30 Apr 2025 06:44:06 GMT"} })
82442025-04-30T06:44:06.687ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0/component/sp/caboose?firmware_slot=1
82452025-04-30T06:44:06.691ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "4687e242-9b65-43c6-94cb-f649decdf0e7", "content-length": "111", "date": "Wed, 30 Apr 2025 06:44:06 GMT"} })
82462025-04-30T06:44:06.691ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0/component/rot/caboose?firmware_slot=0
82472025-04-30T06:44:06.694ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "3540ea39-1ecd-47a5-8471-df45af4deefb", "content-length": "170", "date": "Wed, 30 Apr 2025 06:44:06 GMT"} })
82482025-04-30T06:44:06.694ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0/component/rot/caboose?firmware_slot=1
82492025-04-30T06:44:06.699ZDEBGtest_utilization: attempt to fetch latest inventory collection result = Ok("found")
82502025-04-30T06:44:06.706ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "1fb2606a-176b-47d5-9527-74a41f60d009", "content-length": "170", "date": "Wed, 30 Apr 2025 06:44:06 GMT"} })
82512025-04-30T06:44:06.706ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0/component/stage0/caboose?firmware_slot=0
82522025-04-30T06:44:06.711ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "e63e051d-7b82-4dfa-be0f-54b4f238eaaa", "content-length": "179", "date": "Wed, 30 Apr 2025 06:44:06 GMT"} })
82532025-04-30T06:44:06.712ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0/component/stage0/caboose?firmware_slot=1
82542025-04-30T06:44:06.716ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "68376da1-c126-4906-b69c-f2814419c7e0", "content-length": "179", "date": "Wed, 30 Apr 2025 06:44:06 GMT"} })
82552025-04-30T06:44:06.716ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0/component/rot/cmpa
82562025-04-30T06:44:06.717ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "17f7a989-4355-4211-bb4d-19a93b5380e7", "content-length": "702", "date": "Wed, 30 Apr 2025 06:44:06 GMT"} })
82572025-04-30T06:44:06.717ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0/component/rot/cfpa
82582025-04-30T06:44:06.719ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "c3fd293c-4703-41ae-b632-ca9fc4c8b5db", "content-length": "727", "date": "Wed, 30 Apr 2025 06:44:06 GMT"} })
82592025-04-30T06:44:06.719ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0/component/rot/cfpa
82602025-04-30T06:44:06.720ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "3d01f644-3e1b-4ace-bc58-d737762c639e", "content-length": "729", "date": "Wed, 30 Apr 2025 06:44:06 GMT"} })
82612025-04-30T06:44:06.720ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0/component/rot/cfpa
82622025-04-30T06:44:06.722ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "b629dbd3-bb4f-495f-9977-a727bd109c80", "content-length": "728", "date": "Wed, 30 Apr 2025 06:44:06 GMT"} })
82632025-04-30T06:44:06.722ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1
82642025-04-30T06:44:06.723ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1", status: 200, headers: {"content-type": "application/json", "x-request-id": "4f848687-e62c-4323-9280-0e240d21192c", "content-length": "734", "date": "Wed, 30 Apr 2025 06:44:06 GMT"} })
82652025-04-30T06:44:06.725ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1/component/sp/caboose?firmware_slot=0
82662025-04-30T06:44:06.738ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "4fcbad2d-9900-432f-9239-0daa4d67cea2", "content-length": "111", "date": "Wed, 30 Apr 2025 06:44:06 GMT"} })
82672025-04-30T06:44:06.739ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1/component/sp/caboose?firmware_slot=1
82682025-04-30T06:44:06.742ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "9c72f986-96d0-45c7-948e-47bde885f09a", "content-length": "111", "date": "Wed, 30 Apr 2025 06:44:06 GMT"} })
82692025-04-30T06:44:06.742ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1/component/rot/caboose?firmware_slot=0
82702025-04-30T06:44:06.746ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "dea79f31-6b33-4da2-b620-540598aebee5", "content-length": "170", "date": "Wed, 30 Apr 2025 06:44:06 GMT"} })
82712025-04-30T06:44:06.746ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1/component/rot/caboose?firmware_slot=1
82722025-04-30T06:44:06.749ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "a21161ee-ca59-49f9-942a-c2d122e5b0fa", "content-length": "170", "date": "Wed, 30 Apr 2025 06:44:06 GMT"} })
82732025-04-30T06:44:06.749ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1/component/stage0/caboose?firmware_slot=0
82742025-04-30T06:44:06.750ZTRACtest_utilization (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory
82752025-04-30T06:44:06.750ZTRACtest_utilization (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
82762025-04-30T06:44:06.750ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
82772025-04-30T06:44:06.750ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
82782025-04-30T06:44:06.753ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
82792025-04-30T06:44:06.757ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
82802025-04-30T06:44:06.777ZDEBGtest_utilization (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory result = Ok(())
82812025-04-30T06:44:06.777ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
82822025-04-30T06:44:06.777ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
82832025-04-30T06:44:06.778ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
82842025-04-30T06:44:06.778ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "8fbbdba0-2d1c-40f0-a045-9ea02927d8a0", "content-length": "179", "date": "Wed, 30 Apr 2025 06:44:06 GMT"} })
82852025-04-30T06:44:06.778ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1/component/stage0/caboose?firmware_slot=1
82862025-04-30T06:44:06.782ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "695d8ee4-d1b8-405f-940a-b764b9385aa8", "content-length": "179", "date": "Wed, 30 Apr 2025 06:44:06 GMT"} })
82872025-04-30T06:44:06.782ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1/component/rot/cmpa
82882025-04-30T06:44:06.783ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
82892025-04-30T06:44:06.783ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
82902025-04-30T06:44:06.783ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
82912025-04-30T06:44:06.784ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "255230d6-016f-4a59-95a4-1c6f5c935c35", "content-length": "702", "date": "Wed, 30 Apr 2025 06:44:06 GMT"} })
82922025-04-30T06:44:06.784ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1/component/rot/cfpa
82932025-04-30T06:44:06.785ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "548dc231-0475-4c90-a8fa-68e5afa877b6", "content-length": "727", "date": "Wed, 30 Apr 2025 06:44:06 GMT"} })
82942025-04-30T06:44:06.786ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1/component/rot/cfpa
82952025-04-30T06:44:06.787ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "0bd80ff8-7826-4517-9ace-69909a838803", "content-length": "729", "date": "Wed, 30 Apr 2025 06:44:06 GMT"} })
82962025-04-30T06:44:06.787ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1/component/rot/cfpa
82972025-04-30T06:44:06.788ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "f68998df-dda2-4f86-bba2-e1cf86cebe32", "content-length": "728", "date": "Wed, 30 Apr 2025 06:44:06 GMT"} })
82982025-04-30T06:44:06.788ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0
82992025-04-30T06:44:06.790ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "87dc0e27-e397-4aea-a800-aa1cd4c28431", "content-length": "723", "date": "Wed, 30 Apr 2025 06:44:06 GMT"} })
83002025-04-30T06:44:06.790ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0/component/sp/caboose?firmware_slot=0
83012025-04-30T06:44:06.799ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "56a95254-06ca-4abd-8d62-c34db5f08063", "content-length": "109", "date": "Wed, 30 Apr 2025 06:44:06 GMT"} })
83022025-04-30T06:44:06.799ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0/component/sp/caboose?firmware_slot=1
83032025-04-30T06:44:06.808ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "e3a29b5d-3a38-4557-9180-bafc17710850", "content-length": "109", "date": "Wed, 30 Apr 2025 06:44:06 GMT"} })
83042025-04-30T06:44:06.808ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0/component/rot/caboose?firmware_slot=0
83052025-04-30T06:44:06.812ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "5562d5bd-5534-4d17-9902-4db64599fe46", "content-length": "169", "date": "Wed, 30 Apr 2025 06:44:06 GMT"} })
83062025-04-30T06:44:06.812ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0/component/rot/caboose?firmware_slot=1
83072025-04-30T06:44:06.816ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "10ea1ddd-89a2-49f2-a911-2e8d63492e0d", "content-length": "169", "date": "Wed, 30 Apr 2025 06:44:06 GMT"} })
83082025-04-30T06:44:06.816ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0/component/stage0/caboose?firmware_slot=0
83092025-04-30T06:44:06.818ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "4f3b00d5-08d4-4b42-b8d1-7e2ff94c0de0", "content-length": "178", "date": "Wed, 30 Apr 2025 06:44:06 GMT"} })
83102025-04-30T06:44:06.818ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0/component/stage0/caboose?firmware_slot=1
83112025-04-30T06:44:06.822ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "713169ce-88e4-4750-9449-1964a962a445", "content-length": "178", "date": "Wed, 30 Apr 2025 06:44:06 GMT"} })
83122025-04-30T06:44:06.823ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0/component/rot/cmpa
83132025-04-30T06:44:06.828ZDEBGtest_utilization: attempt to fetch latest inventory collection result = Ok("found")
83142025-04-30T06:44:06.829ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "525156f4-fb2f-4fd9-813c-4e5727c1c8ed", "content-length": "702", "date": "Wed, 30 Apr 2025 06:44:06 GMT"} })
83152025-04-30T06:44:06.829ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0/component/rot/cfpa
83162025-04-30T06:44:06.831ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "dbaa731f-291a-48be-b390-53bba7be72af", "content-length": "727", "date": "Wed, 30 Apr 2025 06:44:06 GMT"} })
83172025-04-30T06:44:06.831ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0/component/rot/cfpa
83182025-04-30T06:44:06.835ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "acc4b96f-0847-49c8-9bbb-c48083ad4b91", "content-length": "729", "date": "Wed, 30 Apr 2025 06:44:06 GMT"} })
83192025-04-30T06:44:06.835ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0/component/rot/cfpa
83202025-04-30T06:44:06.836ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "6f005057-4c10-403e-90de-2be09b362485", "content-length": "728", "date": "Wed, 30 Apr 2025 06:44:06 GMT"} })
83212025-04-30T06:44:06.836ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1
83222025-04-30T06:44:06.878ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1", status: 200, headers: {"content-type": "application/json", "x-request-id": "aa430e4a-c738-4077-a7b0-94851ba9a1d7", "content-length": "723", "date": "Wed, 30 Apr 2025 06:44:06 GMT"} })
83232025-04-30T06:44:06.879ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1/component/sp/caboose?firmware_slot=0
83242025-04-30T06:44:06.880ZTRACtest_utilization (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory
83252025-04-30T06:44:06.880ZTRACtest_utilization (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
83262025-04-30T06:44:06.880ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
83272025-04-30T06:44:06.880ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
83282025-04-30T06:44:06.882ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
83292025-04-30T06:44:06.887ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
83302025-04-30T06:44:06.902ZDEBGtest_utilization (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory result = Ok(())
83312025-04-30T06:44:06.902ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
83322025-04-30T06:44:06.902ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
83332025-04-30T06:44:06.908ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
83342025-04-30T06:44:06.913ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
83352025-04-30T06:44:06.913ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
83362025-04-30T06:44:06.914ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
83372025-04-30T06:44:06.916ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "30efbfb1-98f9-4058-8fa6-9e5b9145288f", "content-length": "109", "date": "Wed, 30 Apr 2025 06:44:06 GMT"} })
83382025-04-30T06:44:06.916ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1/component/sp/caboose?firmware_slot=1
83392025-04-30T06:44:06.918ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "7bd7fed7-04d6-4416-9e33-91ee4ddbedbf", "content-length": "109", "date": "Wed, 30 Apr 2025 06:44:06 GMT"} })
83402025-04-30T06:44:06.919ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1/component/rot/caboose?firmware_slot=0
83412025-04-30T06:44:06.925ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "be969108-ef52-4d37-8508-bbb135b8afbe", "content-length": "169", "date": "Wed, 30 Apr 2025 06:44:06 GMT"} })
83422025-04-30T06:44:06.925ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1/component/rot/caboose?firmware_slot=1
83432025-04-30T06:44:06.930ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "7b47fafb-a4bf-48b0-9fcd-e2127d112488", "content-length": "169", "date": "Wed, 30 Apr 2025 06:44:06 GMT"} })
83442025-04-30T06:44:06.930ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1/component/stage0/caboose?firmware_slot=0
83452025-04-30T06:44:06.933ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "cdc8e4ec-8df1-4039-a593-11390fa87698", "content-length": "178", "date": "Wed, 30 Apr 2025 06:44:06 GMT"} })
83462025-04-30T06:44:06.933ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1/component/stage0/caboose?firmware_slot=1
83472025-04-30T06:44:06.936ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "901b90b0-25b7-4a27-bee6-83edee6629f3", "content-length": "178", "date": "Wed, 30 Apr 2025 06:44:06 GMT"} })
83482025-04-30T06:44:06.936ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1/component/rot/cmpa
83492025-04-30T06:44:06.937ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "aff4998e-dddb-4c94-9353-f389f5d0d53c", "content-length": "702", "date": "Wed, 30 Apr 2025 06:44:06 GMT"} })
83502025-04-30T06:44:06.938ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1/component/rot/cfpa
83512025-04-30T06:44:06.940ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "5117716d-ff69-4291-bc4a-eb9dea4c1ba8", "content-length": "727", "date": "Wed, 30 Apr 2025 06:44:06 GMT"} })
83522025-04-30T06:44:06.940ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1/component/rot/cfpa
83532025-04-30T06:44:06.942ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "3b6a853d-cd88-4992-b1c5-60b1eddcb30d", "content-length": "729", "date": "Wed, 30 Apr 2025 06:44:06 GMT"} })
83542025-04-30T06:44:06.942ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1/component/rot/cfpa
83552025-04-30T06:44:06.943ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "9abd0300-a01c-4d5c-8b0e-b7f7d4997297", "content-length": "728", "date": "Wed, 30 Apr 2025 06:44:06 GMT"} })
83562025-04-30T06:44:06.944ZTRACtest_utilization (BackgroundTasks): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet
83572025-04-30T06:44:06.944ZTRACtest_utilization (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
83582025-04-30T06:44:06.944ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
83592025-04-30T06:44:06.944ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
83602025-04-30T06:44:06.945ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
83612025-04-30T06:44:06.949ZDEBGtest_utilization: attempt to fetch latest inventory collection result = Ok("found")
83622025-04-30T06:44:06.949ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
83632025-04-30T06:44:06.954ZDEBGtest_utilization (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet result = Ok(())
83642025-04-30T06:44:06.954ZTRACtest_utilization (BackgroundTasks): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet
83652025-04-30T06:44:06.954ZTRACtest_utilization (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
83662025-04-30T06:44:06.954ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
83672025-04-30T06:44:06.954ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
83682025-04-30T06:44:06.954ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
83692025-04-30T06:44:06.957ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
83702025-04-30T06:44:06.961ZDEBGtest_utilization (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet result = Ok(())
83712025-04-30T06:44:06.961ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
83722025-04-30T06:44:06.961ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
83732025-04-30T06:44:06.962ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
83742025-04-30T06:44:07.038ZDEBGtest_utilization (BackgroundTasks): begin collection from Sled Agent background_task = inventory_collection sled_agent_url = http://[::1]:39365
83752025-04-30T06:44:07.038ZDEBGtest_utilization (BackgroundTasks): client request SledAgent = http://[::1]:39365 background_task = inventory_collection body = None method = GET uri = http://[::1]:39365/inventory
83762025-04-30T06:44:07.038ZTRACtest_utilization (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory
83772025-04-30T06:44:07.038ZTRACtest_utilization (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
83782025-04-30T06:44:07.038ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
83792025-04-30T06:44:07.039ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
83802025-04-30T06:44:07.040ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
83812025-04-30T06:44:07.040ZINFOtest_utilization (dropshot): accepted connection local_addr = [::1]:39365 remote_addr = [::1]:53760 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
83822025-04-30T06:44:07.040ZDEBGtest_utilization (collection-task): sent timer-based collection request to the collection task collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
83832025-04-30T06:44:07.041ZDEBGtest_utilization (collection-task): collecting from producer collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
83842025-04-30T06:44:07.042ZTRACtest_utilization (dropshot): incoming request local_addr = [::1]:39365 method = GET remote_addr = [::1]:53760 req_id = 39059f30-e017-4861-a785-f54ba04f0521 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /inventory
83852025-04-30T06:44:07.044ZINFOtest_utilization (dropshot): request completed latency_us = 1697 local_addr = [::1]:39365 method = GET remote_addr = [::1]:53760 req_id = 39059f30-e017-4861-a785-f54ba04f0521 response_code = 200 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /inventory
83862025-04-30T06:44:07.044ZTRACtest_utilization (collection-task): sent collection request to producer collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
83872025-04-30T06:44:07.045ZDEBGtest_utilization (collection-task): collected results from producer collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 n_results = 1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
83882025-04-30T06:44:07.045ZTRACtest_utilization (collection-task): forwarded results to main collection loop collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
83892025-04-30T06:44:07.045ZTRACtest_utilization (collection-task): top of inner collection loop, waiting for next request collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
83902025-04-30T06:44:07.045ZDEBGtest_utilization (BackgroundTasks): client response SledAgent = http://[::1]:39365 background_task = inventory_collection result = Ok(Response { url: "http://[::1]:39365/inventory", status: 200, headers: {"content-type": "application/json", "x-request-id": "39059f30-e017-4861-a785-f54ba04f0521", "content-length": "3141", "date": "Wed, 30 Apr 2025 06:44:06 GMT"} })
83912025-04-30T06:44:07.049ZDEBGtest_utilization (BackgroundTasks): begin collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
83922025-04-30T06:44:07.049ZDEBGtest_utilization (BackgroundTasks): end collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
83932025-04-30T06:44:07.049ZDEBGtest_utilization (BackgroundTasks): finished collection background_task = inventory_collection
83942025-04-30T06:44:07.051ZTRACtest_utilization (BackgroundTasks): authorize begin action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory
83952025-04-30T06:44:07.051ZTRACtest_utilization (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
83962025-04-30T06:44:07.051ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
83972025-04-30T06:44:07.051ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
83982025-04-30T06:44:07.052ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
83992025-04-30T06:44:07.053ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
84002025-04-30T06:44:07.062ZDEBGtest_utilization (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory result = Ok(())
84012025-04-30T06:44:07.062ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
84022025-04-30T06:44:07.062ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
84032025-04-30T06:44:07.063ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
84042025-04-30T06:44:07.067ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
84052025-04-30T06:44:07.072ZDEBGtest_utilization (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory result = Ok(())
84062025-04-30T06:44:07.072ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
84072025-04-30T06:44:07.072ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
84082025-04-30T06:44:07.074ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
84092025-04-30T06:44:07.076ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
84102025-04-30T06:44:07.077ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
84112025-04-30T06:44:07.077ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
84122025-04-30T06:44:07.114ZWARNtest_utilization (datastore): Non-retryable transaction failure err = DatabaseError(UniqueViolation, "duplicate key value violates unique constraint \\"inv_caboose_pkey\\"") transaction = inventory_insert_collection
84132025-04-30T06:44:07.114ZWARNtest_utilization (BackgroundTasks): inventory collection failed background_task = inventory_collection error = failed to collect inventory: saving inventory to database: Internal Error: unexpected database error: duplicate key value violates unique constraint "inv_caboose_pkey"
84142025-04-30T06:44:07.114ZDEBGtest_utilization (BackgroundTasks): activation complete background_task = inventory_collection elapsed = 601.507344ms iteration = 14 status = {"error":"failed to collect inventory: saving inventory to database: Internal Error: unexpected database error: duplicate key value violates unique constraint \\"inv_caboose_pkey\\""}
84152025-04-30T06:44:07.114ZDEBGtest_utilization (BackgroundTasks): activating background_task = inventory_collection iteration = 15 reason = Signaled
84162025-04-30T06:44:07.114ZTRACtest_utilization (BackgroundTasks): authorize begin action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory
84172025-04-30T06:44:07.114ZTRACtest_utilization (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
84182025-04-30T06:44:07.114ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
84192025-04-30T06:44:07.114ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
84202025-04-30T06:44:07.116ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
84212025-04-30T06:44:07.120ZDEBGtest_utilization: attempt to fetch latest inventory collection result = Ok("found")
84222025-04-30T06:44:07.120ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
84232025-04-30T06:44:07.129ZDEBGtest_utilization (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory result = Ok(())
84242025-04-30T06:44:07.129ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
84252025-04-30T06:44:07.129ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
84262025-04-30T06:44:07.130ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
84272025-04-30T06:44:07.137ZDEBGtest_utilization (BackgroundTasks): inventory_prune_one: nothing eligible for removal (too few) background_task = inventory_collection candidates = [(13d8b8d9-6c6b-47a6-9190-952544edb75b, 0)]
84282025-04-30T06:44:07.137ZTRACtest_utilization (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal
84292025-04-30T06:44:07.138ZTRACtest_utilization (dns): buffer buffer = Length: 67 (0x43) bytes\n0000: 4f f9 01 00 00 01 00 00 00 00 00 01 04 5f 6d 67 O............_mg\n0010: 73 04 5f 74 63 70 0d 63 6f 6e 74 72 6f 6c 2d 70 s._tcp.control-p\n0020: 6c 61 6e 65 05 6f 78 69 64 65 08 69 6e 74 65 72 lane.oxide.inter\n0030: 6e 61 6c 00 00 21 00 01 00 00 29 04 d0 00 00 00 nal..!....).....\n0040: 00 00 00 ... kind = dns peer_addr = [::1]:53262 req_id = 5402fc0e-3509-4934-bed9-ab7edb6b2c10
84302025-04-30T06:44:07.139ZDEBGtest_utilization (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 20473,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_mgs._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_mgs._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 4,\n 95,\n 109,\n 103,\n 115,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:53262 req_id = 5402fc0e-3509-4934-bed9-ab7edb6b2c10
84312025-04-30T06:44:07.139ZDEBGtest_utilization (store): query key key = _mgs._tcp
84322025-04-30T06:44:07.140ZDEBGtest_utilization (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
84332025-04-30T06:44:07.140ZDEBGtest_utilization (dns): dns response additional_records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal"), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] kind = dns peer_addr = [::1]:53262 query = LowerQuery { name: LowerName(Name("_mgs._tcp.control-plane.oxide.internal.")), original: Query { name: Name("_mgs._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN } } records = [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 48759, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal") })) }] req_id = 5402fc0e-3509-4934-bed9-ab7edb6b2c10
84342025-04-30T06:44:07.141ZDEBGtest_utilization (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgs._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 48759, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }], valid_until: Instant { tv_sec: 4311, tv_nsec: 940033039 } })
84352025-04-30T06:44:07.141ZTRACtest_utilization (DnsResolver): lookup_service_targets: looking up SRV target
84362025-04-30T06:44:07.148ZTRACtest_utilization (dns): buffer buffer = Length: 108 (0x6c) bytes\n0000: ce 57 01 00 00 01 00 00 00 00 00 01 2d 64 65 6e .W..........-den\n0010: 64 72 69 74 65 2d 62 36 64 36 35 33 34 31 2d 31 drite-b6d65341-1\n0020: 36 37 63 2d 34 31 64 66 2d 39 62 35 63 2d 34 31 67c-41df-9b5c-41\n0030: 63 64 65 64 39 39 63 32 32 39 04 68 6f 73 74 0d cded99c229.host.\n0040: 63 6f 6e 74 72 6f 6c 2d 70 6c 61 6e 65 05 6f 78 control-plane.ox\n0050: 69 64 65 08 69 6e 74 65 72 6e 61 6c 00 00 1c 00 ide.internal....\n0060: 01 00 00 29 04 d0 00 00 00 00 00 00 ...)........ kind = dns peer_addr = [::1]:60493 req_id = 9f08929f-f182-4890-92e2-f125b239c2bb
84372025-04-30T06:44:07.149ZDEBGtest_utilization (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 52823,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n query_type: AAAA,\n query_class: IN,\n },\n },\n original: [\n 45,\n 100,\n 101,\n 110,\n 100,\n 114,\n 105,\n 116,\n 101,\n 45,\n 98,\n 54,\n 100,\n 54,\n 53,\n 51,\n 52,\n 49,\n 45,\n 49,\n 54,\n 55,\n 99,\n 45,\n 52,\n 49,\n 100,\n 102,\n 45,\n 57,\n 98,\n 53,\n 99,\n 45,\n 52,\n 49,\n 99,\n 100,\n 101,\n 100,\n 57,\n 57,\n 99,\n 50,\n 50,\n 57,\n 4,\n 104,\n 111,\n 115,\n 116,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 28,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:60493 req_id = 9f08929f-f182-4890-92e2-f125b239c2bb
84382025-04-30T06:44:07.149ZDEBGtest_utilization (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
84392025-04-30T06:44:07.149ZDEBGtest_utilization (dns): dns response additional_records = [] kind = dns peer_addr = [::1]:60493 query = LowerQuery { name: LowerName(Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.")), original: Query { name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), query_type: AAAA, query_class: IN } } records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] req_id = 9f08929f-f182-4890-92e2-f125b239c2bb
84402025-04-30T06:44:07.224ZTRACtest_utilization (DnsResolver): lookup_all_socket_v6 srv dns_name = _clickhouse-admin-keeper._tcp.control-plane.oxide.internal
84412025-04-30T06:44:07.224ZTRACtest_utilization (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory
84422025-04-30T06:44:07.225ZTRACtest_utilization (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
84432025-04-30T06:44:07.225ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
84442025-04-30T06:44:07.225ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
84452025-04-30T06:44:07.226ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
84462025-04-30T06:44:07.226ZTRACtest_utilization (dns): buffer buffer = Length: 87 (0x57) bytes\n0000: 48 29 01 00 00 01 00 00 00 00 00 01 18 5f 63 6c H)..........._cl\n0010: 69 63 6b 68 6f 75 73 65 2d 61 64 6d 69 6e 2d 6b ickhouse-admin-k\n0020: 65 65 70 65 72 04 5f 74 63 70 0d 63 6f 6e 74 72 eeper._tcp.contr\n0030: 6f 6c 2d 70 6c 61 6e 65 05 6f 78 69 64 65 08 69 ol-plane.oxide.i\n0040: 6e 74 65 72 6e 61 6c 00 00 21 00 01 00 00 29 04 nternal..!....).\n0050: d0 00 00 00 00 00 00 ....... kind = dns peer_addr = [::1]:64816 req_id = 23f1f36e-ba2b-4964-9f5b-6fa6182e18fd
84472025-04-30T06:44:07.227ZDEBGtest_utilization (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 18473,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 24,\n 95,\n 99,\n 108,\n 105,\n 99,\n 107,\n 104,\n 111,\n 117,\n 115,\n 101,\n 45,\n 97,\n 100,\n 109,\n 105,\n 110,\n 45,\n 107,\n 101,\n 101,\n 112,\n 101,\n 114,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:64816 req_id = 23f1f36e-ba2b-4964-9f5b-6fa6182e18fd
84482025-04-30T06:44:07.227ZDEBGtest_utilization (store): query key key = _clickhouse-admin-keeper._tcp
84492025-04-30T06:44:07.227ZERROtest_utilization (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 18473, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 1, }, query: WireQuery { query: LowerQuery { name: LowerName( Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), ), original: Query { name: Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN, }, }, original: [ 24, 95, 99, 108, 105, 99, 107, 104, 111, 117, 115, 101, 45, 97, 100, 109, 105, 110, 45, 107, 101, 101, 112, 101, 114, 4, 95, 116, 99, 112, 13, 99, 111, 110, 116, 114, 111, 108, 45, 112, 108, 97, 110, 101, 5, 111, 120, 105, 100, 101, 8, 105, 110, 116, 101, 114, 110, 97, 108, 0, 0, 33, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: Some( Edns { rcode_high: 0, version: 0, dnssec_ok: false, max_payload: 1232, options: OPT { options: {}, }, }, ), } NXDOMAIN: no records found for name: "_clickhouse-admin-keeper._tcp.control-plane.oxide.internal." kind = dns peer_addr = [::1]:64816 req_id = 23f1f36e-ba2b-4964-9f5b-6fa6182e18fd
84502025-04-30T06:44:07.228ZDEBGtest_utilization (BackgroundTasks): begin collection background_task = inventory_collection
84512025-04-30T06:44:07.229ZDEBGtest_utilization (BackgroundTasks): begin collection from MGS background_task = inventory_collection mgs_url = http://[::1]:48759
84522025-04-30T06:44:07.229ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/ignition
84532025-04-30T06:44:07.229ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
84542025-04-30T06:44:07.235ZDEBGtest_utilization (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory result = Ok(())
84552025-04-30T06:44:07.236ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
84562025-04-30T06:44:07.236ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
84572025-04-30T06:44:07.237ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
84582025-04-30T06:44:07.242ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
84592025-04-30T06:44:07.242ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
84602025-04-30T06:44:07.244ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
84612025-04-30T06:44:07.250ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/ignition", status: 200, headers: {"content-type": "application/json", "x-request-id": "4f768a8d-ce2e-44c5-89d3-f7fef5d3facb", "content-length": "835", "date": "Wed, 30 Apr 2025 06:44:06 GMT"} })
84622025-04-30T06:44:07.251ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0
84632025-04-30T06:44:07.253ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "9ab8a489-1940-44c0-83cf-602493b32162", "content-length": "734", "date": "Wed, 30 Apr 2025 06:44:06 GMT"} })
84642025-04-30T06:44:07.254ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0/component/sp/caboose?firmware_slot=0
84652025-04-30T06:44:07.257ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "a030bc25-0968-48ec-8dfc-0664cab4cc6c", "content-length": "111", "date": "Wed, 30 Apr 2025 06:44:06 GMT"} })
84662025-04-30T06:44:07.257ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0/component/sp/caboose?firmware_slot=1
84672025-04-30T06:44:07.260ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "6a8df183-1e70-4911-aab7-9b304c56b480", "content-length": "111", "date": "Wed, 30 Apr 2025 06:44:06 GMT"} })
84682025-04-30T06:44:07.260ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0/component/rot/caboose?firmware_slot=0
84692025-04-30T06:44:07.264ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "48faac6b-7a02-4b1d-9159-d30daf51e45b", "content-length": "170", "date": "Wed, 30 Apr 2025 06:44:06 GMT"} })
84702025-04-30T06:44:07.265ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0/component/rot/caboose?firmware_slot=1
84712025-04-30T06:44:07.269ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "03c758fc-fdff-43bb-9cfd-f0c75f0d6ad2", "content-length": "170", "date": "Wed, 30 Apr 2025 06:44:06 GMT"} })
84722025-04-30T06:44:07.269ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0/component/stage0/caboose?firmware_slot=0
84732025-04-30T06:44:07.269ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "2d0b633d-6f35-492f-89f9-2f8b1ca755fd", "content-length": "179", "date": "Wed, 30 Apr 2025 06:44:06 GMT"} })
84742025-04-30T06:44:07.269ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0/component/stage0/caboose?firmware_slot=1
84752025-04-30T06:44:07.275ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "b4a76173-ec7b-4071-a9e5-098f040b1b3c", "content-length": "179", "date": "Wed, 30 Apr 2025 06:44:06 GMT"} })
84762025-04-30T06:44:07.275ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0/component/rot/cmpa
84772025-04-30T06:44:07.281ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "92fe6c42-ed80-4993-9f6a-82462ba4da6b", "content-length": "702", "date": "Wed, 30 Apr 2025 06:44:06 GMT"} })
84782025-04-30T06:44:07.282ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0/component/rot/cfpa
84792025-04-30T06:44:07.285ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "aa5d6b66-e00e-4532-8276-2bd816843169", "content-length": "727", "date": "Wed, 30 Apr 2025 06:44:06 GMT"} })
84802025-04-30T06:44:07.286ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0/component/rot/cfpa
84812025-04-30T06:44:07.291ZDEBGtest_utilization: attempt to fetch latest inventory collection result = Ok("found")
84822025-04-30T06:44:07.307ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "cc54e49e-a28e-45b8-931d-aa9b3eea9dc9", "content-length": "729", "date": "Wed, 30 Apr 2025 06:44:06 GMT"} })
84832025-04-30T06:44:07.307ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0/component/rot/cfpa
84842025-04-30T06:44:07.310ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "d75cec8a-5b60-439d-886e-6e332c0c1081", "content-length": "728", "date": "Wed, 30 Apr 2025 06:44:06 GMT"} })
84852025-04-30T06:44:07.310ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1
84862025-04-30T06:44:07.313ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1", status: 200, headers: {"content-type": "application/json", "x-request-id": "77d742fb-d69f-4b3b-ae2c-1506a714a5cc", "content-length": "734", "date": "Wed, 30 Apr 2025 06:44:06 GMT"} })
84872025-04-30T06:44:07.313ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1/component/sp/caboose?firmware_slot=0
84882025-04-30T06:44:07.317ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "2c15a192-5721-4bc2-ab39-8a9262d80c67", "content-length": "111", "date": "Wed, 30 Apr 2025 06:44:06 GMT"} })
84892025-04-30T06:44:07.317ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1/component/sp/caboose?firmware_slot=1
84902025-04-30T06:44:07.325ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "3d19186e-70a9-494f-b908-45f2272295bd", "content-length": "111", "date": "Wed, 30 Apr 2025 06:44:06 GMT"} })
84912025-04-30T06:44:07.325ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1/component/rot/caboose?firmware_slot=0
84922025-04-30T06:44:07.329ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "5da4436b-98e4-41cf-80a8-4c00994b05ae", "content-length": "170", "date": "Wed, 30 Apr 2025 06:44:06 GMT"} })
84932025-04-30T06:44:07.329ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1/component/rot/caboose?firmware_slot=1
84942025-04-30T06:44:07.334ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "29fc052a-90af-4b05-80aa-10f785580a7f", "content-length": "170", "date": "Wed, 30 Apr 2025 06:44:06 GMT"} })
84952025-04-30T06:44:07.334ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1/component/stage0/caboose?firmware_slot=0
84962025-04-30T06:44:07.339ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "16e4ed2e-1774-4f61-9c54-53f96440cbd2", "content-length": "179", "date": "Wed, 30 Apr 2025 06:44:06 GMT"} })
84972025-04-30T06:44:07.340ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1/component/stage0/caboose?firmware_slot=1
84982025-04-30T06:44:07.341ZTRACtest_utilization (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory
84992025-04-30T06:44:07.341ZTRACtest_utilization (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
85002025-04-30T06:44:07.341ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
85012025-04-30T06:44:07.341ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
85022025-04-30T06:44:07.343ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
85032025-04-30T06:44:07.347ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "edc69b7f-c97a-42b2-b752-c43f3dfe3339", "content-length": "179", "date": "Wed, 30 Apr 2025 06:44:06 GMT"} })
85042025-04-30T06:44:07.347ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1/component/rot/cmpa
85052025-04-30T06:44:07.348ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
85062025-04-30T06:44:07.355ZDEBGtest_utilization (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory result = Ok(())
85072025-04-30T06:44:07.355ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
85082025-04-30T06:44:07.355ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
85092025-04-30T06:44:07.356ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
85102025-04-30T06:44:07.357ZDEBGtest_utilization (results-sink): inserting 1 samples into database collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
85112025-04-30T06:44:07.357ZDEBGtest_utilization (results-sink-cluster): inserting 1 samples into database collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
85122025-04-30T06:44:07.357ZDEBGtest_utilization (clickhouse-client): unrolling 1 total samples collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 4d1485f0-110e-4161-9a33-c4610d68be5d
85132025-04-30T06:44:07.359ZTRACtest_utilization (clickhouse-client): inserting data collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 4d1485f0-110e-4161-9a33-c4610d68be5d n_columns = 4 n_rows = 2 sql = INSERT INTO oximeter.fields_string FORMAT Native
85142025-04-30T06:44:07.361ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
85152025-04-30T06:44:07.361ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
85162025-04-30T06:44:07.363ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
85172025-04-30T06:44:07.364ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "fcda3a80-51ee-4c97-a624-2728d9a19193", "content-length": "702", "date": "Wed, 30 Apr 2025 06:44:06 GMT"} })
85182025-04-30T06:44:07.364ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1/component/rot/cfpa
85192025-04-30T06:44:07.366ZDEBGtest_utilization (clickhouse-client): inserted rows into table collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 4d1485f0-110e-4161-9a33-c4610d68be5d n_rows = 2 table_name = fields_string
85202025-04-30T06:44:07.366ZTRACtest_utilization (clickhouse-client): inserting data collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 4d1485f0-110e-4161-9a33-c4610d68be5d n_columns = 4 n_rows = 1 sql = INSERT INTO oximeter.measurements_i64 FORMAT Native
85212025-04-30T06:44:07.367ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "08f5ab0b-8886-45d7-a863-03cf789318cf", "content-length": "727", "date": "Wed, 30 Apr 2025 06:44:06 GMT"} })
85222025-04-30T06:44:07.367ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1/component/rot/cfpa
85232025-04-30T06:44:07.369ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "f724f63a-1f9d-43bb-aece-f62ebc2a7bb5", "content-length": "729", "date": "Wed, 30 Apr 2025 06:44:06 GMT"} })
85242025-04-30T06:44:07.369ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1/component/rot/cfpa
85252025-04-30T06:44:07.370ZDEBGtest_utilization (clickhouse-client): inserted rows into table collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 4d1485f0-110e-4161-9a33-c4610d68be5d n_rows = 1 table_name = measurements_i64
85262025-04-30T06:44:07.370ZTRACtest_utilization (results-sink): successfully inserted samples collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
85272025-04-30T06:44:07.372ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "733eb845-2430-4246-8b7c-d5b5c7b59d29", "content-length": "728", "date": "Wed, 30 Apr 2025 06:44:06 GMT"} })
85282025-04-30T06:44:07.372ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0
85292025-04-30T06:44:07.375ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "74f9d1ee-28ae-4978-a953-a56b398fd867", "content-length": "723", "date": "Wed, 30 Apr 2025 06:44:06 GMT"} })
85302025-04-30T06:44:07.375ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0/component/sp/caboose?firmware_slot=0
85312025-04-30T06:44:07.380ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "537077e4-6ab0-4178-b27d-770c8f15ed3c", "content-length": "109", "date": "Wed, 30 Apr 2025 06:44:06 GMT"} })
85322025-04-30T06:44:07.380ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0/component/sp/caboose?firmware_slot=1
85332025-04-30T06:44:07.385ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "c893fb21-2e6c-46c5-a21d-739fa6f808c8", "content-length": "109", "date": "Wed, 30 Apr 2025 06:44:06 GMT"} })
85342025-04-30T06:44:07.385ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0/component/rot/caboose?firmware_slot=0
85352025-04-30T06:44:07.397ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "7c10b8ff-68a0-4dad-95a1-3863255b0c77", "content-length": "169", "date": "Wed, 30 Apr 2025 06:44:06 GMT"} })
85362025-04-30T06:44:07.397ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0/component/rot/caboose?firmware_slot=1
85372025-04-30T06:44:07.404ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "de305af6-4562-4ca8-8da6-195e8b239d3f", "content-length": "169", "date": "Wed, 30 Apr 2025 06:44:06 GMT"} })
85382025-04-30T06:44:07.405ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0/component/stage0/caboose?firmware_slot=0
85392025-04-30T06:44:07.406ZDEBGtest_utilization: attempt to fetch latest inventory collection result = Ok("found")
85402025-04-30T06:44:07.410ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "c272cec7-1aaa-4a3b-9561-e8400d9699df", "content-length": "178", "date": "Wed, 30 Apr 2025 06:44:06 GMT"} })
85412025-04-30T06:44:07.411ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0/component/stage0/caboose?firmware_slot=1
85422025-04-30T06:44:07.417ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "72ff0ddf-eebf-40b0-8956-da33835cb6b1", "content-length": "178", "date": "Wed, 30 Apr 2025 06:44:06 GMT"} })
85432025-04-30T06:44:07.417ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0/component/rot/cmpa
85442025-04-30T06:44:07.419ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "b51b0d01-70df-4e25-8bd9-4227f827ec94", "content-length": "702", "date": "Wed, 30 Apr 2025 06:44:06 GMT"} })
85452025-04-30T06:44:07.420ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0/component/rot/cfpa
85462025-04-30T06:44:07.422ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "4e13f57c-98d5-4307-a949-32ab21616df4", "content-length": "727", "date": "Wed, 30 Apr 2025 06:44:06 GMT"} })
85472025-04-30T06:44:07.422ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0/component/rot/cfpa
85482025-04-30T06:44:07.423ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "1975fc67-14d7-429a-a843-f8f567d45048", "content-length": "729", "date": "Wed, 30 Apr 2025 06:44:06 GMT"} })
85492025-04-30T06:44:07.425ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0/component/rot/cfpa
85502025-04-30T06:44:07.427ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "f0cdc293-e08e-4c08-b300-b17399167a14", "content-length": "728", "date": "Wed, 30 Apr 2025 06:44:06 GMT"} })
85512025-04-30T06:44:07.427ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1
85522025-04-30T06:44:07.428ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1", status: 200, headers: {"content-type": "application/json", "x-request-id": "feaecd9f-86bc-4b60-9901-cec6ecdf9d11", "content-length": "723", "date": "Wed, 30 Apr 2025 06:44:06 GMT"} })
85532025-04-30T06:44:07.428ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1/component/sp/caboose?firmware_slot=0
85542025-04-30T06:44:07.431ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "a834602f-3fa2-4f2b-99ec-83b380fc5ad4", "content-length": "109", "date": "Wed, 30 Apr 2025 06:44:06 GMT"} })
85552025-04-30T06:44:07.431ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1/component/sp/caboose?firmware_slot=1
85562025-04-30T06:44:07.434ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "2ac419d0-114c-4986-a40e-18b85277c9a3", "content-length": "109", "date": "Wed, 30 Apr 2025 06:44:06 GMT"} })
85572025-04-30T06:44:07.434ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1/component/rot/caboose?firmware_slot=0
85582025-04-30T06:44:07.437ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "826a3b91-fca8-4b8c-a05e-292e711807f1", "content-length": "169", "date": "Wed, 30 Apr 2025 06:44:06 GMT"} })
85592025-04-30T06:44:07.437ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1/component/rot/caboose?firmware_slot=1
85602025-04-30T06:44:07.439ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "58472cee-0946-4533-bbcd-f52b212ec6b0", "content-length": "169", "date": "Wed, 30 Apr 2025 06:44:06 GMT"} })
85612025-04-30T06:44:07.439ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1/component/stage0/caboose?firmware_slot=0
85622025-04-30T06:44:07.442ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "68ee5b38-27ab-455f-9cf6-f526b020a2a3", "content-length": "178", "date": "Wed, 30 Apr 2025 06:44:06 GMT"} })
85632025-04-30T06:44:07.442ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1/component/stage0/caboose?firmware_slot=1
85642025-04-30T06:44:07.446ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "f3b8a24c-a848-4aef-a200-9b5222d0587b", "content-length": "178", "date": "Wed, 30 Apr 2025 06:44:06 GMT"} })
85652025-04-30T06:44:07.446ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1/component/rot/cmpa
85662025-04-30T06:44:07.448ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "b4e9f8f9-24c9-4963-870c-9e6ee563c4b4", "content-length": "702", "date": "Wed, 30 Apr 2025 06:44:06 GMT"} })
85672025-04-30T06:44:07.448ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1/component/rot/cfpa
85682025-04-30T06:44:07.450ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "1e4e091b-5115-455a-bcbb-dff4745f75cc", "content-length": "727", "date": "Wed, 30 Apr 2025 06:44:06 GMT"} })
85692025-04-30T06:44:07.451ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1/component/rot/cfpa
85702025-04-30T06:44:07.452ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "6b31e1d6-f6af-4f60-8fef-06c6afd67a39", "content-length": "729", "date": "Wed, 30 Apr 2025 06:44:06 GMT"} })
85712025-04-30T06:44:07.452ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1/component/rot/cfpa
85722025-04-30T06:44:07.454ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "8ca57134-5450-4527-bbef-1dd20094b497", "content-length": "728", "date": "Wed, 30 Apr 2025 06:44:06 GMT"} })
85732025-04-30T06:44:07.454ZTRACtest_utilization (BackgroundTasks): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet
85742025-04-30T06:44:07.454ZTRACtest_utilization (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
85752025-04-30T06:44:07.454ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
85762025-04-30T06:44:07.454ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
85772025-04-30T06:44:07.456ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
85782025-04-30T06:44:07.458ZTRACtest_utilization (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory
85792025-04-30T06:44:07.458ZTRACtest_utilization (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
85802025-04-30T06:44:07.458ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
85812025-04-30T06:44:07.458ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
85822025-04-30T06:44:07.461ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
85832025-04-30T06:44:07.462ZWARNtest_utilization (results-sink-cluster): failed to insert some results into metric DB collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 error = Failed to check out connection to database: No backends found for this service file = oximeter/collector/src/results_sink.rs:92
85842025-04-30T06:44:07.464ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
85852025-04-30T06:44:07.469ZDEBGtest_utilization (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet result = Ok(())
85862025-04-30T06:44:07.469ZTRACtest_utilization (BackgroundTasks): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet
85872025-04-30T06:44:07.469ZTRACtest_utilization (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
85882025-04-30T06:44:07.469ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
85892025-04-30T06:44:07.469ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
85902025-04-30T06:44:07.470ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
85912025-04-30T06:44:07.472ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
85922025-04-30T06:44:07.479ZDEBGtest_utilization (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory result = Ok(())
85932025-04-30T06:44:07.479ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
85942025-04-30T06:44:07.479ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
85952025-04-30T06:44:07.479ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
85962025-04-30T06:44:07.480ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
85972025-04-30T06:44:07.486ZDEBGtest_utilization (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet result = Ok(())
85982025-04-30T06:44:07.486ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
85992025-04-30T06:44:07.486ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
86002025-04-30T06:44:07.486ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
86012025-04-30T06:44:07.488ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
86022025-04-30T06:44:07.488ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
86032025-04-30T06:44:07.489ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
86042025-04-30T06:44:07.548ZDEBGtest_utilization: attempt to fetch latest inventory collection result = Ok("found")
86052025-04-30T06:44:07.627ZDEBGtest_utilization (BackgroundTasks): begin collection from Sled Agent background_task = inventory_collection sled_agent_url = http://[::1]:39365
86062025-04-30T06:44:07.628ZDEBGtest_utilization (BackgroundTasks): client request SledAgent = http://[::1]:39365 background_task = inventory_collection body = None method = GET uri = http://[::1]:39365/inventory
86072025-04-30T06:44:07.629ZTRACtest_utilization (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory
86082025-04-30T06:44:07.629ZTRACtest_utilization (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
86092025-04-30T06:44:07.629ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
86102025-04-30T06:44:07.629ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
86112025-04-30T06:44:07.631ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
86122025-04-30T06:44:07.631ZINFOtest_utilization (dropshot): accepted connection local_addr = [::1]:39365 remote_addr = [::1]:45428 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
86132025-04-30T06:44:07.632ZTRACtest_utilization (dropshot): incoming request local_addr = [::1]:39365 method = GET remote_addr = [::1]:45428 req_id = b11afab1-a686-4899-9f97-6b8712bd9b12 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /inventory
86142025-04-30T06:44:07.636ZINFOtest_utilization (dropshot): request completed latency_us = 4314 local_addr = [::1]:39365 method = GET remote_addr = [::1]:45428 req_id = b11afab1-a686-4899-9f97-6b8712bd9b12 response_code = 200 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /inventory
86152025-04-30T06:44:07.637ZDEBGtest_utilization (BackgroundTasks): client response SledAgent = http://[::1]:39365 background_task = inventory_collection result = Ok(Response { url: "http://[::1]:39365/inventory", status: 200, headers: {"content-type": "application/json", "x-request-id": "b11afab1-a686-4899-9f97-6b8712bd9b12", "content-length": "3141", "date": "Wed, 30 Apr 2025 06:44:06 GMT"} })
86162025-04-30T06:44:07.640ZDEBGtest_utilization (BackgroundTasks): begin collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
86172025-04-30T06:44:07.640ZDEBGtest_utilization (BackgroundTasks): end collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
86182025-04-30T06:44:07.640ZDEBGtest_utilization (BackgroundTasks): finished collection background_task = inventory_collection
86192025-04-30T06:44:07.644ZTRACtest_utilization (BackgroundTasks): authorize begin action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory
86202025-04-30T06:44:07.644ZTRACtest_utilization (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
86212025-04-30T06:44:07.644ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
86222025-04-30T06:44:07.645ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
86232025-04-30T06:44:07.647ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
86242025-04-30T06:44:07.647ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
86252025-04-30T06:44:07.653ZDEBGtest_utilization (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory result = Ok(())
86262025-04-30T06:44:07.653ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
86272025-04-30T06:44:07.653ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
86282025-04-30T06:44:07.654ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
86292025-04-30T06:44:07.661ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
86302025-04-30T06:44:07.668ZDEBGtest_utilization (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory result = Ok(())
86312025-04-30T06:44:07.669ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
86322025-04-30T06:44:07.669ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
86332025-04-30T06:44:07.669ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
86342025-04-30T06:44:07.672ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
86352025-04-30T06:44:07.672ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
86362025-04-30T06:44:07.673ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
86372025-04-30T06:44:07.729ZWARNtest_utilization (datastore): Non-retryable transaction failure err = DatabaseError(UniqueViolation, "duplicate key value violates unique constraint \\"inv_caboose_pkey\\"") transaction = inventory_insert_collection
86382025-04-30T06:44:07.729ZWARNtest_utilization (BackgroundTasks): inventory collection failed background_task = inventory_collection error = failed to collect inventory: saving inventory to database: Internal Error: unexpected database error: duplicate key value violates unique constraint "inv_caboose_pkey"
86392025-04-30T06:44:07.729ZDEBGtest_utilization (BackgroundTasks): activation complete background_task = inventory_collection elapsed = 614.692705ms iteration = 15 status = {"error":"failed to collect inventory: saving inventory to database: Internal Error: unexpected database error: duplicate key value violates unique constraint \\"inv_caboose_pkey\\""}
86402025-04-30T06:44:07.729ZDEBGtest_utilization (BackgroundTasks): activating background_task = inventory_collection iteration = 16 reason = Signaled
86412025-04-30T06:44:07.729ZTRACtest_utilization (BackgroundTasks): authorize begin action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory
86422025-04-30T06:44:07.729ZTRACtest_utilization (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
86432025-04-30T06:44:07.729ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
86442025-04-30T06:44:07.729ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
86452025-04-30T06:44:07.731ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
86462025-04-30T06:44:07.736ZDEBGtest_utilization: attempt to fetch latest inventory collection result = Ok("found")
86472025-04-30T06:44:07.737ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
86482025-04-30T06:44:07.752ZDEBGtest_utilization (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory result = Ok(())
86492025-04-30T06:44:07.752ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
86502025-04-30T06:44:07.752ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
86512025-04-30T06:44:07.753ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
86522025-04-30T06:44:07.760ZDEBGtest_utilization (BackgroundTasks): inventory_prune_one: nothing eligible for removal (too few) background_task = inventory_collection candidates = [(13d8b8d9-6c6b-47a6-9190-952544edb75b, 0)]
86532025-04-30T06:44:07.763ZTRACtest_utilization (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal
86542025-04-30T06:44:07.764ZTRACtest_utilization (dns): buffer buffer = Length: 67 (0x43) bytes\n0000: ce 1c 01 00 00 01 00 00 00 00 00 01 04 5f 6d 67 ............._mg\n0010: 73 04 5f 74 63 70 0d 63 6f 6e 74 72 6f 6c 2d 70 s._tcp.control-p\n0020: 6c 61 6e 65 05 6f 78 69 64 65 08 69 6e 74 65 72 lane.oxide.inter\n0030: 6e 61 6c 00 00 21 00 01 00 00 29 04 d0 00 00 00 nal..!....).....\n0040: 00 00 00 ... kind = dns peer_addr = [::1]:54367 req_id = 71baed9f-5226-499c-a4a0-8de9747ce9eb
86552025-04-30T06:44:07.764ZDEBGtest_utilization (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 52764,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_mgs._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_mgs._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 4,\n 95,\n 109,\n 103,\n 115,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:54367 req_id = 71baed9f-5226-499c-a4a0-8de9747ce9eb
86562025-04-30T06:44:07.766ZDEBGtest_utilization (store): query key key = _mgs._tcp
86572025-04-30T06:44:07.766ZDEBGtest_utilization (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
86582025-04-30T06:44:07.766ZDEBGtest_utilization (dns): dns response additional_records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal"), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] kind = dns peer_addr = [::1]:54367 query = LowerQuery { name: LowerName(Name("_mgs._tcp.control-plane.oxide.internal.")), original: Query { name: Name("_mgs._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN } } records = [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 48759, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal") })) }] req_id = 71baed9f-5226-499c-a4a0-8de9747ce9eb
86592025-04-30T06:44:07.767ZDEBGtest_utilization (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgs._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 48759, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }], valid_until: Instant { tv_sec: 4312, tv_nsec: 566314755 } })
86602025-04-30T06:44:07.767ZTRACtest_utilization (DnsResolver): lookup_service_targets: looking up SRV target
86612025-04-30T06:44:07.768ZTRACtest_utilization (dns): buffer buffer = Length: 108 (0x6c) bytes\n0000: 4f c2 01 00 00 01 00 00 00 00 00 01 2d 64 65 6e O...........-den\n0010: 64 72 69 74 65 2d 62 36 64 36 35 33 34 31 2d 31 drite-b6d65341-1\n0020: 36 37 63 2d 34 31 64 66 2d 39 62 35 63 2d 34 31 67c-41df-9b5c-41\n0030: 63 64 65 64 39 39 63 32 32 39 04 68 6f 73 74 0d cded99c229.host.\n0040: 63 6f 6e 74 72 6f 6c 2d 70 6c 61 6e 65 05 6f 78 control-plane.ox\n0050: 69 64 65 08 69 6e 74 65 72 6e 61 6c 00 00 1c 00 ide.internal....\n0060: 01 00 00 29 04 d0 00 00 00 00 00 00 ...)........ kind = dns peer_addr = [::1]:63508 req_id = 9b3a9fef-6e56-4302-a76d-f3a4d4e20b8e
86622025-04-30T06:44:07.768ZDEBGtest_utilization (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 20418,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n query_type: AAAA,\n query_class: IN,\n },\n },\n original: [\n 45,\n 100,\n 101,\n 110,\n 100,\n 114,\n 105,\n 116,\n 101,\n 45,\n 98,\n 54,\n 100,\n 54,\n 53,\n 51,\n 52,\n 49,\n 45,\n 49,\n 54,\n 55,\n 99,\n 45,\n 52,\n 49,\n 100,\n 102,\n 45,\n 57,\n 98,\n 53,\n 99,\n 45,\n 52,\n 49,\n 99,\n 100,\n 101,\n 100,\n 57,\n 57,\n 99,\n 50,\n 50,\n 57,\n 4,\n 104,\n 111,\n 115,\n 116,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 28,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:63508 req_id = 9b3a9fef-6e56-4302-a76d-f3a4d4e20b8e
86632025-04-30T06:44:07.768ZDEBGtest_utilization (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
86642025-04-30T06:44:07.768ZDEBGtest_utilization (dns): dns response additional_records = [] kind = dns peer_addr = [::1]:63508 query = LowerQuery { name: LowerName(Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.")), original: Query { name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), query_type: AAAA, query_class: IN } } records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] req_id = 9b3a9fef-6e56-4302-a76d-f3a4d4e20b8e
86652025-04-30T06:44:07.837ZTRACtest_utilization (DnsResolver): lookup_all_socket_v6 srv dns_name = _clickhouse-admin-keeper._tcp.control-plane.oxide.internal
86662025-04-30T06:44:07.838ZTRACtest_utilization (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory
86672025-04-30T06:44:07.838ZTRACtest_utilization (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
86682025-04-30T06:44:07.838ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
86692025-04-30T06:44:07.838ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
86702025-04-30T06:44:07.839ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
86712025-04-30T06:44:07.839ZTRACtest_utilization (dns): buffer buffer = Length: 87 (0x57) bytes\n0000: d4 8b 01 00 00 01 00 00 00 00 00 01 18 5f 63 6c ............._cl\n0010: 69 63 6b 68 6f 75 73 65 2d 61 64 6d 69 6e 2d 6b ickhouse-admin-k\n0020: 65 65 70 65 72 04 5f 74 63 70 0d 63 6f 6e 74 72 eeper._tcp.contr\n0030: 6f 6c 2d 70 6c 61 6e 65 05 6f 78 69 64 65 08 69 ol-plane.oxide.i\n0040: 6e 74 65 72 6e 61 6c 00 00 21 00 01 00 00 29 04 nternal..!....).\n0050: d0 00 00 00 00 00 00 ....... kind = dns peer_addr = [::1]:52823 req_id = 5d1870ef-c506-4d72-9ad8-58188b550e22
86722025-04-30T06:44:07.839ZDEBGtest_utilization (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 54411,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 24,\n 95,\n 99,\n 108,\n 105,\n 99,\n 107,\n 104,\n 111,\n 117,\n 115,\n 101,\n 45,\n 97,\n 100,\n 109,\n 105,\n 110,\n 45,\n 107,\n 101,\n 101,\n 112,\n 101,\n 114,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:52823 req_id = 5d1870ef-c506-4d72-9ad8-58188b550e22
86732025-04-30T06:44:07.840ZDEBGtest_utilization (store): query key key = _clickhouse-admin-keeper._tcp
86742025-04-30T06:44:07.840ZERROtest_utilization (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 54411, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 1, }, query: WireQuery { query: LowerQuery { name: LowerName( Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), ), original: Query { name: Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN, }, }, original: [ 24, 95, 99, 108, 105, 99, 107, 104, 111, 117, 115, 101, 45, 97, 100, 109, 105, 110, 45, 107, 101, 101, 112, 101, 114, 4, 95, 116, 99, 112, 13, 99, 111, 110, 116, 114, 111, 108, 45, 112, 108, 97, 110, 101, 5, 111, 120, 105, 100, 101, 8, 105, 110, 116, 101, 114, 110, 97, 108, 0, 0, 33, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: Some( Edns { rcode_high: 0, version: 0, dnssec_ok: false, max_payload: 1232, options: OPT { options: {}, }, }, ), } NXDOMAIN: no records found for name: "_clickhouse-admin-keeper._tcp.control-plane.oxide.internal." kind = dns peer_addr = [::1]:52823 req_id = 5d1870ef-c506-4d72-9ad8-58188b550e22
86752025-04-30T06:44:07.840ZDEBGtest_utilization (BackgroundTasks): begin collection background_task = inventory_collection
86762025-04-30T06:44:07.840ZDEBGtest_utilization (BackgroundTasks): begin collection from MGS background_task = inventory_collection mgs_url = http://[::1]:48759
86772025-04-30T06:44:07.840ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/ignition
86782025-04-30T06:44:07.842ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
86792025-04-30T06:44:07.863ZDEBGtest_utilization (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory result = Ok(())
86802025-04-30T06:44:07.863ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
86812025-04-30T06:44:07.863ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
86822025-04-30T06:44:07.864ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
86832025-04-30T06:44:07.866ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/ignition", status: 200, headers: {"content-type": "application/json", "x-request-id": "315c80e7-b367-41b0-b437-2abe9c52864e", "content-length": "835", "date": "Wed, 30 Apr 2025 06:44:07 GMT"} })
86842025-04-30T06:44:07.868ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0
86852025-04-30T06:44:07.868ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
86862025-04-30T06:44:07.868ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
86872025-04-30T06:44:07.869ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
86882025-04-30T06:44:07.873ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "94193b10-2a84-48d5-be9d-7a1835d6fc2a", "content-length": "734", "date": "Wed, 30 Apr 2025 06:44:07 GMT"} })
86892025-04-30T06:44:07.873ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0/component/sp/caboose?firmware_slot=0
86902025-04-30T06:44:07.879ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "c2727a2c-edff-4fcb-9fe5-b63732e69b70", "content-length": "111", "date": "Wed, 30 Apr 2025 06:44:07 GMT"} })
86912025-04-30T06:44:07.879ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0/component/sp/caboose?firmware_slot=1
86922025-04-30T06:44:07.882ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "80d44ebe-e95a-43ff-81ea-bc91be88c832", "content-length": "111", "date": "Wed, 30 Apr 2025 06:44:07 GMT"} })
86932025-04-30T06:44:07.882ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0/component/rot/caboose?firmware_slot=0
86942025-04-30T06:44:07.888ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "98f619de-a52c-4130-b436-ce9d4a197164", "content-length": "170", "date": "Wed, 30 Apr 2025 06:44:07 GMT"} })
86952025-04-30T06:44:07.889ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0/component/rot/caboose?firmware_slot=1
86962025-04-30T06:44:07.896ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "18d9b2eb-c34a-491d-a86c-03c79e2c7070", "content-length": "170", "date": "Wed, 30 Apr 2025 06:44:07 GMT"} })
86972025-04-30T06:44:07.896ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0/component/stage0/caboose?firmware_slot=0
86982025-04-30T06:44:07.903ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "06bc1c11-31f5-4501-8bf7-bc409cfacea4", "content-length": "179", "date": "Wed, 30 Apr 2025 06:44:07 GMT"} })
86992025-04-30T06:44:07.903ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0/component/stage0/caboose?firmware_slot=1
87002025-04-30T06:44:07.914ZDEBGtest_utilization: attempt to fetch latest inventory collection result = Ok("found")
87012025-04-30T06:44:07.914ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "d9a85b25-b9c2-4ab9-8da3-f5555a28c1f8", "content-length": "179", "date": "Wed, 30 Apr 2025 06:44:07 GMT"} })
87022025-04-30T06:44:07.915ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0/component/rot/cmpa
87032025-04-30T06:44:07.917ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "ff3dafca-61bf-41a0-b4f6-b7c3c8a494cb", "content-length": "702", "date": "Wed, 30 Apr 2025 06:44:07 GMT"} })
87042025-04-30T06:44:07.917ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0/component/rot/cfpa
87052025-04-30T06:44:07.920ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "52adb4e5-c8ad-4ed3-a884-56e2d57bf807", "content-length": "727", "date": "Wed, 30 Apr 2025 06:44:07 GMT"} })
87062025-04-30T06:44:07.921ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0/component/rot/cfpa
87072025-04-30T06:44:07.923ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "fc7df9c6-6b05-48a8-8af9-4895663d6e89", "content-length": "729", "date": "Wed, 30 Apr 2025 06:44:07 GMT"} })
87082025-04-30T06:44:07.923ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0/component/rot/cfpa
87092025-04-30T06:44:07.925ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "1eba378f-5eba-4ca1-a19e-c7ad70640d12", "content-length": "728", "date": "Wed, 30 Apr 2025 06:44:07 GMT"} })
87102025-04-30T06:44:07.925ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1
87112025-04-30T06:44:07.928ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1", status: 200, headers: {"content-type": "application/json", "x-request-id": "abfb8f46-6f1d-436c-8393-37ac7edee919", "content-length": "734", "date": "Wed, 30 Apr 2025 06:44:07 GMT"} })
87122025-04-30T06:44:07.928ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1/component/sp/caboose?firmware_slot=0
87132025-04-30T06:44:07.935ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "5b675890-bc47-4a86-82b8-4a34de7501ee", "content-length": "111", "date": "Wed, 30 Apr 2025 06:44:07 GMT"} })
87142025-04-30T06:44:07.936ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1/component/sp/caboose?firmware_slot=1
87152025-04-30T06:44:07.938ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "65f560ac-5ffc-4260-92ef-6ee52ca62dd9", "content-length": "111", "date": "Wed, 30 Apr 2025 06:44:07 GMT"} })
87162025-04-30T06:44:07.939ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1/component/rot/caboose?firmware_slot=0
87172025-04-30T06:44:07.942ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "344b284a-810a-4c52-9bc2-e661a86ab146", "content-length": "170", "date": "Wed, 30 Apr 2025 06:44:07 GMT"} })
87182025-04-30T06:44:07.942ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1/component/rot/caboose?firmware_slot=1
87192025-04-30T06:44:07.946ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "00d4aab5-27c6-44ae-a78f-1fa9346e343b", "content-length": "170", "date": "Wed, 30 Apr 2025 06:44:07 GMT"} })
87202025-04-30T06:44:07.946ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1/component/stage0/caboose?firmware_slot=0
87212025-04-30T06:44:07.948ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "2b583d9d-7410-40e1-9dfb-a348589fca9d", "content-length": "179", "date": "Wed, 30 Apr 2025 06:44:07 GMT"} })
87222025-04-30T06:44:07.949ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1/component/stage0/caboose?firmware_slot=1
87232025-04-30T06:44:07.951ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "1d85f987-4f47-4e99-8717-a3e26cc849a6", "content-length": "179", "date": "Wed, 30 Apr 2025 06:44:07 GMT"} })
87242025-04-30T06:44:07.951ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1/component/rot/cmpa
87252025-04-30T06:44:07.952ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "970b8088-efd8-4f4a-a759-30fac6efb686", "content-length": "702", "date": "Wed, 30 Apr 2025 06:44:07 GMT"} })
87262025-04-30T06:44:07.952ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1/component/rot/cfpa
87272025-04-30T06:44:07.953ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "6b206238-85f4-4b99-afc2-74128e533f34", "content-length": "727", "date": "Wed, 30 Apr 2025 06:44:07 GMT"} })
87282025-04-30T06:44:07.953ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1/component/rot/cfpa
87292025-04-30T06:44:07.954ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "65c3436d-a3d4-442b-a203-7a5258d7a4b9", "content-length": "729", "date": "Wed, 30 Apr 2025 06:44:07 GMT"} })
87302025-04-30T06:44:07.954ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1/component/rot/cfpa
87312025-04-30T06:44:07.955ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "1cf790c3-57f8-4be2-922b-8e77b1204c51", "content-length": "728", "date": "Wed, 30 Apr 2025 06:44:07 GMT"} })
87322025-04-30T06:44:07.955ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0
87332025-04-30T06:44:07.956ZINFOtest_utilization (oximeter-agent): refreshing list of producers from Nexus collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 file = oximeter/collector/src/agent.rs:562
87342025-04-30T06:44:07.956ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "8660fa40-855a-4436-8808-b8f166bdca56", "content-length": "723", "date": "Wed, 30 Apr 2025 06:44:07 GMT"} })
87352025-04-30T06:44:07.957ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0/component/sp/caboose?firmware_slot=0
87362025-04-30T06:44:07.957ZDEBGtest_utilization (oximeter): client request body = None method = GET uri = http://127.0.0.1:32956/metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?sort_by=id_ascending
87372025-04-30T06:44:07.957ZTRACtest_utilization (dropshot_internal): incoming request local_addr = 127.0.0.1:32956 method = GET remote_addr = 127.0.0.1:55797 req_id = ca2b3952-b330-4975-8273-3dd943f68c8d uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?sort_by=id_ascending
87382025-04-30T06:44:07.958ZTRACtest_utilization (dropshot_internal): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = 127.0.0.1:32956 method = GET remote_addr = 127.0.0.1:55797 req_id = ca2b3952-b330-4975-8273-3dd943f68c8d resource = Database uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?sort_by=id_ascending
87392025-04-30T06:44:07.958ZDEBGtest_utilization (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = 127.0.0.1:32956 method = GET remote_addr = 127.0.0.1:55797 req_id = ca2b3952-b330-4975-8273-3dd943f68c8d roles = RoleSet { roles: {} } uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?sort_by=id_ascending
87402025-04-30T06:44:07.958ZDEBGtest_utilization (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = 127.0.0.1:32956 method = GET remote_addr = 127.0.0.1:55797 req_id = ca2b3952-b330-4975-8273-3dd943f68c8d resource = Database result = Ok(()) uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?sort_by=id_ascending
87412025-04-30T06:44:07.960ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "7e6b316b-4714-4c4f-a533-ff7dd33ad9f3", "content-length": "109", "date": "Wed, 30 Apr 2025 06:44:07 GMT"} })
87422025-04-30T06:44:07.960ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0/component/sp/caboose?firmware_slot=1
87432025-04-30T06:44:07.961ZINFOtest_utilization (dropshot_internal): request completed latency_us = 3860 local_addr = 127.0.0.1:32956 method = GET remote_addr = 127.0.0.1:55797 req_id = ca2b3952-b330-4975-8273-3dd943f68c8d response_code = 200 uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?sort_by=id_ascending
87442025-04-30T06:44:07.961ZDEBGtest_utilization (oximeter): client response result = Ok(Response { url: "http://127.0.0.1:32956/metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?sort_by=id_ascending", status: 200, headers: {"content-type": "application/json", "x-request-id": "ca2b3952-b330-4975-8273-3dd943f68c8d", "content-length": "405", "date": "Wed, 30 Apr 2025 06:44:07 GMT"} })
87452025-04-30T06:44:07.962ZDEBGtest_utilization (oximeter): client request body = None method = GET uri = http://127.0.0.1:32956/metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJhYzU1ZDhkOS1hMjk4LTQ1MzQtODk4Yi1iZWVlMWM5ZTRlNTIifX0%3D
87462025-04-30T06:44:07.962ZTRACtest_utilization (dropshot_internal): incoming request local_addr = 127.0.0.1:32956 method = GET remote_addr = 127.0.0.1:55797 req_id = 9bd8c4cb-67c7-4b39-a1df-adbe317c7a69 uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJhYzU1ZDhkOS1hMjk4LTQ1MzQtODk4Yi1iZWVlMWM5ZTRlNTIifX0%3D
87472025-04-30T06:44:07.962ZTRACtest_utilization (dropshot_internal): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = 127.0.0.1:32956 method = GET remote_addr = 127.0.0.1:55797 req_id = 9bd8c4cb-67c7-4b39-a1df-adbe317c7a69 resource = Database uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJhYzU1ZDhkOS1hMjk4LTQ1MzQtODk4Yi1iZWVlMWM5ZTRlNTIifX0%3D
87482025-04-30T06:44:07.962ZDEBGtest_utilization (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = 127.0.0.1:32956 method = GET remote_addr = 127.0.0.1:55797 req_id = 9bd8c4cb-67c7-4b39-a1df-adbe317c7a69 roles = RoleSet { roles: {} } uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJhYzU1ZDhkOS1hMjk4LTQ1MzQtODk4Yi1iZWVlMWM5ZTRlNTIifX0%3D
87492025-04-30T06:44:07.963ZDEBGtest_utilization (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = 127.0.0.1:32956 method = GET remote_addr = 127.0.0.1:55797 req_id = 9bd8c4cb-67c7-4b39-a1df-adbe317c7a69 resource = Database result = Ok(()) uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJhYzU1ZDhkOS1hMjk4LTQ1MzQtODk4Yi1iZWVlMWM5ZTRlNTIifX0%3D
87502025-04-30T06:44:07.964ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "8424fa52-3f61-4a68-b7f3-5cd054bb3a7a", "content-length": "109", "date": "Wed, 30 Apr 2025 06:44:07 GMT"} })
87512025-04-30T06:44:07.964ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0/component/rot/caboose?firmware_slot=0
87522025-04-30T06:44:07.965ZINFOtest_utilization (dropshot_internal): request completed latency_us = 3109 local_addr = 127.0.0.1:32956 method = GET remote_addr = 127.0.0.1:55797 req_id = 9bd8c4cb-67c7-4b39-a1df-adbe317c7a69 response_code = 200 uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJhYzU1ZDhkOS1hMjk4LTQ1MzQtODk4Yi1iZWVlMWM5ZTRlNTIifX0%3D
87532025-04-30T06:44:07.965ZTRACtest_utilization (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory
87542025-04-30T06:44:07.965ZTRACtest_utilization (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
87552025-04-30T06:44:07.965ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
87562025-04-30T06:44:07.965ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
87572025-04-30T06:44:07.966ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
87582025-04-30T06:44:07.966ZDEBGtest_utilization (oximeter): client response result = Ok(Response { url: "http://127.0.0.1:32956/metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJhYzU1ZDhkOS1hMjk4LTQ1MzQtODk4Yi1iZWVlMWM5ZTRlNTIifX0%3D", status: 200, headers: {"content-type": "application/json", "x-request-id": "9bd8c4cb-67c7-4b39-a1df-adbe317c7a69", "content-length": "29", "date": "Wed, 30 Apr 2025 06:44:07 GMT"} })
87592025-04-30T06:44:07.966ZTRACtest_utilization (oximeter-agent): ignoring request to update existing metric producer, since the endpoint information is the same as the existing collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
87602025-04-30T06:44:07.966ZTRACtest_utilization (oximeter-agent): ignoring request to update existing metric producer, since the endpoint information is the same as the existing collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = ac55d8d9-a298-4534-898b-beee1c9e4e52
87612025-04-30T06:44:07.966ZINFOtest_utilization (oximeter-agent): refreshed list of producers from Nexus collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 file = oximeter/collector/src/agent.rs:637 n_current_tasks = 2 n_pruned_tasks = 0
87622025-04-30T06:44:07.968ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "4100f3cb-6cf4-465d-a824-d31ba58201ca", "content-length": "169", "date": "Wed, 30 Apr 2025 06:44:07 GMT"} })
87632025-04-30T06:44:07.968ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0/component/rot/caboose?firmware_slot=1
87642025-04-30T06:44:07.969ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
87652025-04-30T06:44:07.974ZDEBGtest_utilization (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory result = Ok(())
87662025-04-30T06:44:07.974ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
87672025-04-30T06:44:07.974ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
87682025-04-30T06:44:07.974ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
87692025-04-30T06:44:07.977ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
87702025-04-30T06:44:07.977ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
87712025-04-30T06:44:07.979ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
87722025-04-30T06:44:07.979ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "d65008b5-72ab-457b-8eec-fe87d9674386", "content-length": "169", "date": "Wed, 30 Apr 2025 06:44:07 GMT"} })
87732025-04-30T06:44:07.979ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0/component/stage0/caboose?firmware_slot=0
87742025-04-30T06:44:07.981ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "f235b6b4-baf8-4416-9a58-17617baa38fa", "content-length": "178", "date": "Wed, 30 Apr 2025 06:44:07 GMT"} })
87752025-04-30T06:44:07.981ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0/component/stage0/caboose?firmware_slot=1
87762025-04-30T06:44:07.984ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "3445ecbb-4d1d-4509-a8ad-3af8dfcd42ad", "content-length": "178", "date": "Wed, 30 Apr 2025 06:44:07 GMT"} })
87772025-04-30T06:44:07.984ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0/component/rot/cmpa
87782025-04-30T06:44:07.985ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "3f0360c4-b64f-4b7e-9bf3-1ca0bcea9a65", "content-length": "702", "date": "Wed, 30 Apr 2025 06:44:07 GMT"} })
87792025-04-30T06:44:07.985ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0/component/rot/cfpa
87802025-04-30T06:44:07.986ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "9b91a452-0d39-4e5d-b0db-520c8baa375a", "content-length": "727", "date": "Wed, 30 Apr 2025 06:44:07 GMT"} })
87812025-04-30T06:44:07.987ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0/component/rot/cfpa
87822025-04-30T06:44:07.988ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "0e63bcee-fb25-4a79-8e84-2d98ae67aee7", "content-length": "729", "date": "Wed, 30 Apr 2025 06:44:07 GMT"} })
87832025-04-30T06:44:07.988ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0/component/rot/cfpa
87842025-04-30T06:44:07.989ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "cc699220-6dc8-414c-8d0e-3886cb8db68b", "content-length": "728", "date": "Wed, 30 Apr 2025 06:44:07 GMT"} })
87852025-04-30T06:44:07.989ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1
87862025-04-30T06:44:07.991ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1", status: 200, headers: {"content-type": "application/json", "x-request-id": "08ed0086-4b73-4e42-a018-686365f42239", "content-length": "723", "date": "Wed, 30 Apr 2025 06:44:07 GMT"} })
87872025-04-30T06:44:07.991ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1/component/sp/caboose?firmware_slot=0
87882025-04-30T06:44:07.994ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "361b0e85-60c5-4619-80cd-ef11c0192f9a", "content-length": "109", "date": "Wed, 30 Apr 2025 06:44:07 GMT"} })
87892025-04-30T06:44:07.994ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1/component/sp/caboose?firmware_slot=1
87902025-04-30T06:44:07.995ZDEBGtest_utilization (collection-task): sent timer-based collection request to the collection task collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
87912025-04-30T06:44:07.995ZDEBGtest_utilization (collection-task): collecting from producer collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
87922025-04-30T06:44:08.002ZTRACtest_utilization (collection-task): sent collection request to producer collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
87932025-04-30T06:44:08.005ZDEBGtest_utilization (collection-task): collected results from producer collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 n_results = 1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
87942025-04-30T06:44:08.005ZTRACtest_utilization (collection-task): forwarded results to main collection loop collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
87952025-04-30T06:44:08.005ZTRACtest_utilization (collection-task): top of inner collection loop, waiting for next request collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
87962025-04-30T06:44:08.008ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "c975f418-91fa-4fd0-aebf-abefa1b19ca3", "content-length": "109", "date": "Wed, 30 Apr 2025 06:44:07 GMT"} })
87972025-04-30T06:44:08.008ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1/component/rot/caboose?firmware_slot=0
87982025-04-30T06:44:08.011ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "ca5c3884-151e-41de-9c11-a4a4a6ce6d8b", "content-length": "169", "date": "Wed, 30 Apr 2025 06:44:07 GMT"} })
87992025-04-30T06:44:08.011ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1/component/rot/caboose?firmware_slot=1
88002025-04-30T06:44:08.016ZDEBGtest_utilization: attempt to fetch latest inventory collection result = Ok("found")
88012025-04-30T06:44:08.020ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "ec641a7f-5ab3-4515-9a6f-537bd4547df7", "content-length": "169", "date": "Wed, 30 Apr 2025 06:44:07 GMT"} })
88022025-04-30T06:44:08.020ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1/component/stage0/caboose?firmware_slot=0
88032025-04-30T06:44:08.024ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "5946e9fa-dfe6-44a2-8bb2-5c3f9917390c", "content-length": "178", "date": "Wed, 30 Apr 2025 06:44:07 GMT"} })
88042025-04-30T06:44:08.025ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1/component/stage0/caboose?firmware_slot=1
88052025-04-30T06:44:08.029ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "d928e2f9-e49e-45aa-90ac-295111efc860", "content-length": "178", "date": "Wed, 30 Apr 2025 06:44:07 GMT"} })
88062025-04-30T06:44:08.029ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1/component/rot/cmpa
88072025-04-30T06:44:08.031ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "8ee8967e-f0c6-4b1b-9c69-c5c42f6b2501", "content-length": "702", "date": "Wed, 30 Apr 2025 06:44:07 GMT"} })
88082025-04-30T06:44:08.031ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1/component/rot/cfpa
88092025-04-30T06:44:08.032ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "3fe2b7ee-e0e0-4b69-b31b-7a9eaab8fe83", "content-length": "727", "date": "Wed, 30 Apr 2025 06:44:07 GMT"} })
88102025-04-30T06:44:08.032ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1/component/rot/cfpa
88112025-04-30T06:44:08.035ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "3f26cb08-fe7a-4c30-958d-d10c6a7f1089", "content-length": "729", "date": "Wed, 30 Apr 2025 06:44:07 GMT"} })
88122025-04-30T06:44:08.035ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1/component/rot/cfpa
88132025-04-30T06:44:08.036ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "44975358-7b4b-4d45-a709-f909a86858cc", "content-length": "728", "date": "Wed, 30 Apr 2025 06:44:07 GMT"} })
88142025-04-30T06:44:08.036ZTRACtest_utilization (BackgroundTasks): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet
88152025-04-30T06:44:08.036ZTRACtest_utilization (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
88162025-04-30T06:44:08.036ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
88172025-04-30T06:44:08.036ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
88182025-04-30T06:44:08.037ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
88192025-04-30T06:44:08.040ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
88202025-04-30T06:44:08.048ZDEBGtest_utilization (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet result = Ok(())
88212025-04-30T06:44:08.048ZTRACtest_utilization (BackgroundTasks): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet
88222025-04-30T06:44:08.049ZTRACtest_utilization (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
88232025-04-30T06:44:08.049ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
88242025-04-30T06:44:08.049ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
88252025-04-30T06:44:08.049ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
88262025-04-30T06:44:08.054ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
88272025-04-30T06:44:08.058ZDEBGtest_utilization (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet result = Ok(())
88282025-04-30T06:44:08.058ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
88292025-04-30T06:44:08.058ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
88302025-04-30T06:44:08.059ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
88312025-04-30T06:44:08.122ZDEBGtest_utilization (BackgroundTasks): begin collection from Sled Agent background_task = inventory_collection sled_agent_url = http://[::1]:39365
88322025-04-30T06:44:08.122ZDEBGtest_utilization (BackgroundTasks): client request SledAgent = http://[::1]:39365 background_task = inventory_collection body = None method = GET uri = http://[::1]:39365/inventory
88332025-04-30T06:44:08.123ZTRACtest_utilization (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory
88342025-04-30T06:44:08.123ZTRACtest_utilization (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
88352025-04-30T06:44:08.123ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
88362025-04-30T06:44:08.123ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
88372025-04-30T06:44:08.124ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
88382025-04-30T06:44:08.124ZINFOtest_utilization (dropshot): accepted connection local_addr = [::1]:39365 remote_addr = [::1]:50863 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
88392025-04-30T06:44:08.125ZTRACtest_utilization (dropshot): incoming request local_addr = [::1]:39365 method = GET remote_addr = [::1]:50863 req_id = 074ab419-6fc1-417f-9485-f418a37cef25 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /inventory
88402025-04-30T06:44:08.127ZINFOtest_utilization (dropshot): request completed latency_us = 1436 local_addr = [::1]:39365 method = GET remote_addr = [::1]:50863 req_id = 074ab419-6fc1-417f-9485-f418a37cef25 response_code = 200 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /inventory
88412025-04-30T06:44:08.127ZDEBGtest_utilization (BackgroundTasks): client response SledAgent = http://[::1]:39365 background_task = inventory_collection result = Ok(Response { url: "http://[::1]:39365/inventory", status: 200, headers: {"content-type": "application/json", "x-request-id": "074ab419-6fc1-417f-9485-f418a37cef25", "content-length": "3141", "date": "Wed, 30 Apr 2025 06:44:07 GMT"} })
88422025-04-30T06:44:08.129ZDEBGtest_utilization (BackgroundTasks): begin collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
88432025-04-30T06:44:08.129ZDEBGtest_utilization (BackgroundTasks): end collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
88442025-04-30T06:44:08.129ZDEBGtest_utilization (BackgroundTasks): finished collection background_task = inventory_collection
88452025-04-30T06:44:08.131ZTRACtest_utilization (BackgroundTasks): authorize begin action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory
88462025-04-30T06:44:08.131ZTRACtest_utilization (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
88472025-04-30T06:44:08.131ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
88482025-04-30T06:44:08.131ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
88492025-04-30T06:44:08.132ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
88502025-04-30T06:44:08.132ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
88512025-04-30T06:44:08.148ZDEBGtest_utilization (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory result = Ok(())
88522025-04-30T06:44:08.148ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
88532025-04-30T06:44:08.148ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
88542025-04-30T06:44:08.148ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
88552025-04-30T06:44:08.154ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
88562025-04-30T06:44:08.158ZDEBGtest_utilization (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory result = Ok(())
88572025-04-30T06:44:08.159ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
88582025-04-30T06:44:08.159ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
88592025-04-30T06:44:08.160ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
88602025-04-30T06:44:08.163ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
88612025-04-30T06:44:08.163ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
88622025-04-30T06:44:08.164ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
88632025-04-30T06:44:08.197ZDEBGtest_utilization: attempt to fetch latest inventory collection result = Ok("found")
88642025-04-30T06:44:08.205ZWARNtest_utilization (datastore): Non-retryable transaction failure err = DatabaseError(UniqueViolation, "duplicate key value violates unique constraint \\"inv_caboose_pkey\\"") transaction = inventory_insert_collection
88652025-04-30T06:44:08.206ZWARNtest_utilization (BackgroundTasks): inventory collection failed background_task = inventory_collection error = failed to collect inventory: saving inventory to database: Internal Error: unexpected database error: duplicate key value violates unique constraint "inv_caboose_pkey"
88662025-04-30T06:44:08.206ZDEBGtest_utilization (BackgroundTasks): activation complete background_task = inventory_collection elapsed = 476.849291ms iteration = 16 status = {"error":"failed to collect inventory: saving inventory to database: Internal Error: unexpected database error: duplicate key value violates unique constraint \\"inv_caboose_pkey\\""}
88672025-04-30T06:44:08.206ZDEBGtest_utilization (BackgroundTasks): activating background_task = inventory_collection iteration = 17 reason = Signaled
88682025-04-30T06:44:08.206ZTRACtest_utilization (BackgroundTasks): authorize begin action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory
88692025-04-30T06:44:08.207ZTRACtest_utilization (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
88702025-04-30T06:44:08.207ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
88712025-04-30T06:44:08.207ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
88722025-04-30T06:44:08.208ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
88732025-04-30T06:44:08.211ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
88742025-04-30T06:44:08.221ZDEBGtest_utilization (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory result = Ok(())
88752025-04-30T06:44:08.222ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
88762025-04-30T06:44:08.222ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
88772025-04-30T06:44:08.223ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
88782025-04-30T06:44:08.227ZDEBGtest_utilization (BackgroundTasks): inventory_prune_one: nothing eligible for removal (too few) background_task = inventory_collection candidates = [(13d8b8d9-6c6b-47a6-9190-952544edb75b, 0)]
88792025-04-30T06:44:08.227ZTRACtest_utilization (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal
88802025-04-30T06:44:08.228ZTRACtest_utilization (dns): buffer buffer = Length: 67 (0x43) bytes\n0000: 42 79 01 00 00 01 00 00 00 00 00 01 04 5f 6d 67 By..........._mg\n0010: 73 04 5f 74 63 70 0d 63 6f 6e 74 72 6f 6c 2d 70 s._tcp.control-p\n0020: 6c 61 6e 65 05 6f 78 69 64 65 08 69 6e 74 65 72 lane.oxide.inter\n0030: 6e 61 6c 00 00 21 00 01 00 00 29 04 d0 00 00 00 nal..!....).....\n0040: 00 00 00 ... kind = dns peer_addr = [::1]:51119 req_id = f99d1c24-b73d-4011-b74f-333dcd5bff79
88812025-04-30T06:44:08.228ZDEBGtest_utilization (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 17017,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_mgs._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_mgs._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 4,\n 95,\n 109,\n 103,\n 115,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:51119 req_id = f99d1c24-b73d-4011-b74f-333dcd5bff79
88822025-04-30T06:44:08.228ZDEBGtest_utilization (store): query key key = _mgs._tcp
88832025-04-30T06:44:08.230ZDEBGtest_utilization (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
88842025-04-30T06:44:08.231ZDEBGtest_utilization (dns): dns response additional_records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal"), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] kind = dns peer_addr = [::1]:51119 query = LowerQuery { name: LowerName(Name("_mgs._tcp.control-plane.oxide.internal.")), original: Query { name: Name("_mgs._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN } } records = [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 48759, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal") })) }] req_id = f99d1c24-b73d-4011-b74f-333dcd5bff79
88852025-04-30T06:44:08.231ZDEBGtest_utilization (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgs._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 48759, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }], valid_until: Instant { tv_sec: 4313, tv_nsec: 30767867 } })
88862025-04-30T06:44:08.232ZTRACtest_utilization (DnsResolver): lookup_service_targets: looking up SRV target
88872025-04-30T06:44:08.232ZTRACtest_utilization (dns): buffer buffer = Length: 108 (0x6c) bytes\n0000: e5 e1 01 00 00 01 00 00 00 00 00 01 2d 64 65 6e ............-den\n0010: 64 72 69 74 65 2d 62 36 64 36 35 33 34 31 2d 31 drite-b6d65341-1\n0020: 36 37 63 2d 34 31 64 66 2d 39 62 35 63 2d 34 31 67c-41df-9b5c-41\n0030: 63 64 65 64 39 39 63 32 32 39 04 68 6f 73 74 0d cded99c229.host.\n0040: 63 6f 6e 74 72 6f 6c 2d 70 6c 61 6e 65 05 6f 78 control-plane.ox\n0050: 69 64 65 08 69 6e 74 65 72 6e 61 6c 00 00 1c 00 ide.internal....\n0060: 01 00 00 29 04 d0 00 00 00 00 00 00 ...)........ kind = dns peer_addr = [::1]:63653 req_id = 738d4c45-2025-4ffc-a1ae-c04729571e5c
88882025-04-30T06:44:08.237ZDEBGtest_utilization (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 58849,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n query_type: AAAA,\n query_class: IN,\n },\n },\n original: [\n 45,\n 100,\n 101,\n 110,\n 100,\n 114,\n 105,\n 116,\n 101,\n 45,\n 98,\n 54,\n 100,\n 54,\n 53,\n 51,\n 52,\n 49,\n 45,\n 49,\n 54,\n 55,\n 99,\n 45,\n 52,\n 49,\n 100,\n 102,\n 45,\n 57,\n 98,\n 53,\n 99,\n 45,\n 52,\n 49,\n 99,\n 100,\n 101,\n 100,\n 57,\n 57,\n 99,\n 50,\n 50,\n 57,\n 4,\n 104,\n 111,\n 115,\n 116,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 28,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:63653 req_id = 738d4c45-2025-4ffc-a1ae-c04729571e5c
88892025-04-30T06:44:08.237ZDEBGtest_utilization (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
88902025-04-30T06:44:08.237ZDEBGtest_utilization (dns): dns response additional_records = [] kind = dns peer_addr = [::1]:63653 query = LowerQuery { name: LowerName(Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.")), original: Query { name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), query_type: AAAA, query_class: IN } } records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] req_id = 738d4c45-2025-4ffc-a1ae-c04729571e5c
88912025-04-30T06:44:08.316ZTRACtest_utilization (DnsResolver): lookup_all_socket_v6 srv dns_name = _clickhouse-admin-keeper._tcp.control-plane.oxide.internal
88922025-04-30T06:44:08.316ZTRACtest_utilization (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory
88932025-04-30T06:44:08.316ZTRACtest_utilization (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
88942025-04-30T06:44:08.317ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
88952025-04-30T06:44:08.317ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
88962025-04-30T06:44:08.318ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
88972025-04-30T06:44:08.319ZTRACtest_utilization (dns): buffer buffer = Length: 87 (0x57) bytes\n0000: e3 e1 01 00 00 01 00 00 00 00 00 01 18 5f 63 6c ............._cl\n0010: 69 63 6b 68 6f 75 73 65 2d 61 64 6d 69 6e 2d 6b ickhouse-admin-k\n0020: 65 65 70 65 72 04 5f 74 63 70 0d 63 6f 6e 74 72 eeper._tcp.contr\n0030: 6f 6c 2d 70 6c 61 6e 65 05 6f 78 69 64 65 08 69 ol-plane.oxide.i\n0040: 6e 74 65 72 6e 61 6c 00 00 21 00 01 00 00 29 04 nternal..!....).\n0050: d0 00 00 00 00 00 00 ....... kind = dns peer_addr = [::1]:60293 req_id = 2180ea27-e254-45ae-b389-b8407c606830
88982025-04-30T06:44:08.319ZDEBGtest_utilization (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 58337,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 24,\n 95,\n 99,\n 108,\n 105,\n 99,\n 107,\n 104,\n 111,\n 117,\n 115,\n 101,\n 45,\n 97,\n 100,\n 109,\n 105,\n 110,\n 45,\n 107,\n 101,\n 101,\n 112,\n 101,\n 114,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:60293 req_id = 2180ea27-e254-45ae-b389-b8407c606830
88992025-04-30T06:44:08.319ZDEBGtest_utilization (store): query key key = _clickhouse-admin-keeper._tcp
89002025-04-30T06:44:08.319ZERROtest_utilization (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 58337, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 1, }, query: WireQuery { query: LowerQuery { name: LowerName( Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), ), original: Query { name: Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN, }, }, original: [ 24, 95, 99, 108, 105, 99, 107, 104, 111, 117, 115, 101, 45, 97, 100, 109, 105, 110, 45, 107, 101, 101, 112, 101, 114, 4, 95, 116, 99, 112, 13, 99, 111, 110, 116, 114, 111, 108, 45, 112, 108, 97, 110, 101, 5, 111, 120, 105, 100, 101, 8, 105, 110, 116, 101, 114, 110, 97, 108, 0, 0, 33, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: Some( Edns { rcode_high: 0, version: 0, dnssec_ok: false, max_payload: 1232, options: OPT { options: {}, }, }, ), } NXDOMAIN: no records found for name: "_clickhouse-admin-keeper._tcp.control-plane.oxide.internal." kind = dns peer_addr = [::1]:60293 req_id = 2180ea27-e254-45ae-b389-b8407c606830
89012025-04-30T06:44:08.319ZDEBGtest_utilization (BackgroundTasks): begin collection background_task = inventory_collection
89022025-04-30T06:44:08.319ZDEBGtest_utilization (BackgroundTasks): begin collection from MGS background_task = inventory_collection mgs_url = http://[::1]:48759
89032025-04-30T06:44:08.320ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/ignition
89042025-04-30T06:44:08.328ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
89052025-04-30T06:44:08.334ZDEBGtest_utilization (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory result = Ok(())
89062025-04-30T06:44:08.334ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
89072025-04-30T06:44:08.334ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
89082025-04-30T06:44:08.336ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
89092025-04-30T06:44:08.341ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
89102025-04-30T06:44:08.341ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
89112025-04-30T06:44:08.341ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
89122025-04-30T06:44:08.341ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/ignition", status: 200, headers: {"content-type": "application/json", "x-request-id": "bf4f5785-08fb-4964-a2ac-ea31d2fd79b3", "content-length": "835", "date": "Wed, 30 Apr 2025 06:44:07 GMT"} })
89132025-04-30T06:44:08.342ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0
89142025-04-30T06:44:08.347ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "f8b190d7-ca61-4d34-a2de-b1d652eed388", "content-length": "734", "date": "Wed, 30 Apr 2025 06:44:07 GMT"} })
89152025-04-30T06:44:08.348ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0/component/sp/caboose?firmware_slot=0
89162025-04-30T06:44:08.350ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "cf459bc4-949c-47f3-bcfa-c4f369a0bbe6", "content-length": "111", "date": "Wed, 30 Apr 2025 06:44:07 GMT"} })
89172025-04-30T06:44:08.351ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0/component/sp/caboose?firmware_slot=1
89182025-04-30T06:44:08.355ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "26e99694-049e-4914-9b3c-9813d54bd142", "content-length": "111", "date": "Wed, 30 Apr 2025 06:44:07 GMT"} })
89192025-04-30T06:44:08.355ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0/component/rot/caboose?firmware_slot=0
89202025-04-30T06:44:08.358ZDEBGtest_utilization (results-sink): inserting 1 samples into database collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
89212025-04-30T06:44:08.359ZDEBGtest_utilization (results-sink-cluster): inserting 1 samples into database collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
89222025-04-30T06:44:08.359ZDEBGtest_utilization (clickhouse-client): unrolling 1 total samples collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 4d1485f0-110e-4161-9a33-c4610d68be5d
89232025-04-30T06:44:08.360ZTRACtest_utilization (clickhouse-client): inserting data collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 4d1485f0-110e-4161-9a33-c4610d68be5d n_columns = 4 n_rows = 2 sql = INSERT INTO oximeter.fields_string FORMAT Native
89242025-04-30T06:44:08.367ZDEBGtest_utilization (clickhouse-client): inserted rows into table collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 4d1485f0-110e-4161-9a33-c4610d68be5d n_rows = 2 table_name = fields_string
89252025-04-30T06:44:08.367ZTRACtest_utilization (clickhouse-client): inserting data collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 4d1485f0-110e-4161-9a33-c4610d68be5d n_columns = 4 n_rows = 1 sql = INSERT INTO oximeter.measurements_i64 FORMAT Native
89262025-04-30T06:44:08.368ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "9f751fd6-9dd2-4f07-bd35-52c62b6fecc8", "content-length": "170", "date": "Wed, 30 Apr 2025 06:44:07 GMT"} })
89272025-04-30T06:44:08.369ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0/component/rot/caboose?firmware_slot=1
89282025-04-30T06:44:08.373ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "23904da3-4655-400c-99b0-f4dcdaf818cd", "content-length": "170", "date": "Wed, 30 Apr 2025 06:44:07 GMT"} })
89292025-04-30T06:44:08.373ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0/component/stage0/caboose?firmware_slot=0
89302025-04-30T06:44:08.374ZDEBGtest_utilization (clickhouse-client): inserted rows into table collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 4d1485f0-110e-4161-9a33-c4610d68be5d n_rows = 1 table_name = measurements_i64
89312025-04-30T06:44:08.374ZTRACtest_utilization (results-sink): successfully inserted samples collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
89322025-04-30T06:44:08.381ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "43eafd3a-50de-4bed-976c-d6189fbef066", "content-length": "179", "date": "Wed, 30 Apr 2025 06:44:07 GMT"} })
89332025-04-30T06:44:08.381ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0/component/stage0/caboose?firmware_slot=1
89342025-04-30T06:44:08.385ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "5a3706b7-db28-4747-8918-24884812a43f", "content-length": "179", "date": "Wed, 30 Apr 2025 06:44:07 GMT"} })
89352025-04-30T06:44:08.385ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0/component/rot/cmpa
89362025-04-30T06:44:08.390ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "14f98f6c-0487-4a80-a884-8a1d75b43f51", "content-length": "702", "date": "Wed, 30 Apr 2025 06:44:07 GMT"} })
89372025-04-30T06:44:08.390ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0/component/rot/cfpa
89382025-04-30T06:44:08.401ZDEBGtest_utilization: attempt to fetch latest inventory collection result = Ok("found")
89392025-04-30T06:44:08.402ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "6e6c1ef1-e974-4184-81c2-082a9d660970", "content-length": "727", "date": "Wed, 30 Apr 2025 06:44:07 GMT"} })
89402025-04-30T06:44:08.403ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0/component/rot/cfpa
89412025-04-30T06:44:08.408ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "8e7cce00-5053-47d8-a8d4-754e08b5bd13", "content-length": "729", "date": "Wed, 30 Apr 2025 06:44:07 GMT"} })
89422025-04-30T06:44:08.408ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0/component/rot/cfpa
89432025-04-30T06:44:08.410ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "a63c29da-6ddd-4feb-9764-3bbd87c82916", "content-length": "728", "date": "Wed, 30 Apr 2025 06:44:07 GMT"} })
89442025-04-30T06:44:08.411ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1
89452025-04-30T06:44:08.412ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1", status: 200, headers: {"content-type": "application/json", "x-request-id": "9280a5d6-1d9d-4bd9-a69f-fe26ec78b3ef", "content-length": "734", "date": "Wed, 30 Apr 2025 06:44:07 GMT"} })
89462025-04-30T06:44:08.412ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1/component/sp/caboose?firmware_slot=0
89472025-04-30T06:44:08.416ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "27c868c0-fdc2-4ffd-ac97-3b75aeb1689c", "content-length": "111", "date": "Wed, 30 Apr 2025 06:44:07 GMT"} })
89482025-04-30T06:44:08.416ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1/component/sp/caboose?firmware_slot=1
89492025-04-30T06:44:08.418ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "3afbe5a8-3d7e-42a5-b2e0-24c99a7840e9", "content-length": "111", "date": "Wed, 30 Apr 2025 06:44:07 GMT"} })
89502025-04-30T06:44:08.418ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1/component/rot/caboose?firmware_slot=0
89512025-04-30T06:44:08.421ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "9d050f0a-f26e-4f9c-a019-e56cc6a9a526", "content-length": "170", "date": "Wed, 30 Apr 2025 06:44:07 GMT"} })
89522025-04-30T06:44:08.421ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1/component/rot/caboose?firmware_slot=1
89532025-04-30T06:44:08.424ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "ade307ed-28a8-4899-af27-f9ee9f8a4b11", "content-length": "170", "date": "Wed, 30 Apr 2025 06:44:07 GMT"} })
89542025-04-30T06:44:08.424ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1/component/stage0/caboose?firmware_slot=0
89552025-04-30T06:44:08.430ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "33b9f173-8ba1-4f7d-b29f-534f55db9fc2", "content-length": "179", "date": "Wed, 30 Apr 2025 06:44:07 GMT"} })
89562025-04-30T06:44:08.430ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1/component/stage0/caboose?firmware_slot=1
89572025-04-30T06:44:08.432ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "48af5472-e769-4a33-accd-d96c37195f07", "content-length": "179", "date": "Wed, 30 Apr 2025 06:44:07 GMT"} })
89582025-04-30T06:44:08.432ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1/component/rot/cmpa
89592025-04-30T06:44:08.434ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "a2e1f7f8-659a-408e-8caf-a6e2a838cad5", "content-length": "702", "date": "Wed, 30 Apr 2025 06:44:07 GMT"} })
89602025-04-30T06:44:08.434ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1/component/rot/cfpa
89612025-04-30T06:44:08.436ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "9f789227-e3d7-402c-8b7e-1c90fb297f83", "content-length": "727", "date": "Wed, 30 Apr 2025 06:44:07 GMT"} })
89622025-04-30T06:44:08.436ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1/component/rot/cfpa
89632025-04-30T06:44:08.438ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "25bfcec1-2dec-45a5-a348-9c33fe361dca", "content-length": "729", "date": "Wed, 30 Apr 2025 06:44:07 GMT"} })
89642025-04-30T06:44:08.438ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1/component/rot/cfpa
89652025-04-30T06:44:08.439ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "66a2a0f8-d660-4f5a-8d84-8c417d90ccfa", "content-length": "728", "date": "Wed, 30 Apr 2025 06:44:07 GMT"} })
89662025-04-30T06:44:08.440ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0
89672025-04-30T06:44:08.442ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "d2209b5a-c27c-4403-bdf8-e4d1419ac144", "content-length": "723", "date": "Wed, 30 Apr 2025 06:44:07 GMT"} })
89682025-04-30T06:44:08.443ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0/component/sp/caboose?firmware_slot=0
89692025-04-30T06:44:08.445ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "26b3dd9b-e283-4530-9169-75eb86084faf", "content-length": "109", "date": "Wed, 30 Apr 2025 06:44:07 GMT"} })
89702025-04-30T06:44:08.445ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0/component/sp/caboose?firmware_slot=1
89712025-04-30T06:44:08.447ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "57a61b12-0e18-40be-9154-e7fcfb0c67d7", "content-length": "109", "date": "Wed, 30 Apr 2025 06:44:07 GMT"} })
89722025-04-30T06:44:08.447ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0/component/rot/caboose?firmware_slot=0
89732025-04-30T06:44:08.448ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "c549d9cd-3fca-4ce9-bd25-ee9c0d02c12e", "content-length": "169", "date": "Wed, 30 Apr 2025 06:44:07 GMT"} })
89742025-04-30T06:44:08.449ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0/component/rot/caboose?firmware_slot=1
89752025-04-30T06:44:08.450ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "33a027d9-6dda-42f7-acb5-f4b3c1f9720b", "content-length": "169", "date": "Wed, 30 Apr 2025 06:44:07 GMT"} })
89762025-04-30T06:44:08.450ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0/component/stage0/caboose?firmware_slot=0
89772025-04-30T06:44:08.453ZTRACtest_utilization (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory
89782025-04-30T06:44:08.453ZTRACtest_utilization (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
89792025-04-30T06:44:08.453ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
89802025-04-30T06:44:08.453ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
89812025-04-30T06:44:08.454ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
89822025-04-30T06:44:08.454ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "547d12bf-f08d-40db-ba17-d9b2a373fc43", "content-length": "178", "date": "Wed, 30 Apr 2025 06:44:07 GMT"} })
89832025-04-30T06:44:08.455ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0/component/stage0/caboose?firmware_slot=1
89842025-04-30T06:44:08.456ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
89852025-04-30T06:44:08.464ZDEBGtest_utilization (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory result = Ok(())
89862025-04-30T06:44:08.464ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
89872025-04-30T06:44:08.464ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
89882025-04-30T06:44:08.464ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
89892025-04-30T06:44:08.465ZWARNtest_utilization (results-sink-cluster): failed to insert some results into metric DB collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 error = Failed to check out connection to database: No backends found for this service file = oximeter/collector/src/results_sink.rs:92
89902025-04-30T06:44:08.466ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "6d0da97f-b15c-4b99-ad73-46645ac222ed", "content-length": "178", "date": "Wed, 30 Apr 2025 06:44:07 GMT"} })
89912025-04-30T06:44:08.466ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0/component/rot/cmpa
89922025-04-30T06:44:08.467ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
89932025-04-30T06:44:08.467ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
89942025-04-30T06:44:08.467ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
89952025-04-30T06:44:08.468ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "0f7f470d-3b6b-4ab2-86b5-a51578097716", "content-length": "702", "date": "Wed, 30 Apr 2025 06:44:07 GMT"} })
89962025-04-30T06:44:08.468ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0/component/rot/cfpa
89972025-04-30T06:44:08.470ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "b0c74e9e-3ef3-45c3-a9b1-172646298307", "content-length": "727", "date": "Wed, 30 Apr 2025 06:44:07 GMT"} })
89982025-04-30T06:44:08.470ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0/component/rot/cfpa
89992025-04-30T06:44:08.471ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "cf346125-548b-4df0-8f01-2509e23e6ef9", "content-length": "729", "date": "Wed, 30 Apr 2025 06:44:07 GMT"} })
90002025-04-30T06:44:08.471ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0/component/rot/cfpa
90012025-04-30T06:44:08.473ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "8f1a2445-d0b8-4951-8654-3ff0d6e2ecfa", "content-length": "728", "date": "Wed, 30 Apr 2025 06:44:07 GMT"} })
90022025-04-30T06:44:08.473ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1
90032025-04-30T06:44:08.475ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1", status: 200, headers: {"content-type": "application/json", "x-request-id": "4333db7b-1449-4a6c-ad4d-177aa1aab78b", "content-length": "723", "date": "Wed, 30 Apr 2025 06:44:07 GMT"} })
90042025-04-30T06:44:08.476ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1/component/sp/caboose?firmware_slot=0
90052025-04-30T06:44:08.478ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "19fb607a-690b-4802-8ff6-f93018bfa703", "content-length": "109", "date": "Wed, 30 Apr 2025 06:44:07 GMT"} })
90062025-04-30T06:44:08.478ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1/component/sp/caboose?firmware_slot=1
90072025-04-30T06:44:08.487ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "d481ddce-ba13-4866-9d20-eed090ba66c5", "content-length": "109", "date": "Wed, 30 Apr 2025 06:44:07 GMT"} })
90082025-04-30T06:44:08.487ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1/component/rot/caboose?firmware_slot=0
90092025-04-30T06:44:08.492ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "232ebad4-e3a5-4c75-9c60-e8fe57efb169", "content-length": "169", "date": "Wed, 30 Apr 2025 06:44:07 GMT"} })
90102025-04-30T06:44:08.492ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1/component/rot/caboose?firmware_slot=1
90112025-04-30T06:44:08.498ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "9701813e-c0b3-43b9-99d1-c7b99d27c4f1", "content-length": "169", "date": "Wed, 30 Apr 2025 06:44:07 GMT"} })
90122025-04-30T06:44:08.498ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1/component/stage0/caboose?firmware_slot=0
90132025-04-30T06:44:08.504ZDEBGtest_utilization: attempt to fetch latest inventory collection result = Ok("found")
90142025-04-30T06:44:08.506ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "9d6113f5-44c8-4eed-a293-291437dfd804", "content-length": "178", "date": "Wed, 30 Apr 2025 06:44:07 GMT"} })
90152025-04-30T06:44:08.506ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1/component/stage0/caboose?firmware_slot=1
90162025-04-30T06:44:08.510ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "225138f9-1f5d-4126-aec6-beff7a3a31b5", "content-length": "178", "date": "Wed, 30 Apr 2025 06:44:07 GMT"} })
90172025-04-30T06:44:08.510ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1/component/rot/cmpa
90182025-04-30T06:44:08.511ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "cf8e98e3-6ea9-46fd-bc86-83265e06ee9d", "content-length": "702", "date": "Wed, 30 Apr 2025 06:44:07 GMT"} })
90192025-04-30T06:44:08.512ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1/component/rot/cfpa
90202025-04-30T06:44:08.513ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "8b610e7e-8dcc-47ee-a551-3b2b0ecfb473", "content-length": "727", "date": "Wed, 30 Apr 2025 06:44:07 GMT"} })
90212025-04-30T06:44:08.513ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1/component/rot/cfpa
90222025-04-30T06:44:08.514ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "39a9ba1a-03c9-4f27-9f96-219c2f39e6ae", "content-length": "729", "date": "Wed, 30 Apr 2025 06:44:07 GMT"} })
90232025-04-30T06:44:08.514ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1/component/rot/cfpa
90242025-04-30T06:44:08.516ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "70bf9706-2bef-41d6-b4a6-3ae119da227f", "content-length": "728", "date": "Wed, 30 Apr 2025 06:44:07 GMT"} })
90252025-04-30T06:44:08.516ZTRACtest_utilization (BackgroundTasks): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet
90262025-04-30T06:44:08.516ZTRACtest_utilization (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
90272025-04-30T06:44:08.516ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
90282025-04-30T06:44:08.516ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
90292025-04-30T06:44:08.517ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
90302025-04-30T06:44:08.519ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
90312025-04-30T06:44:08.523ZDEBGtest_utilization (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet result = Ok(())
90322025-04-30T06:44:08.523ZTRACtest_utilization (BackgroundTasks): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet
90332025-04-30T06:44:08.523ZTRACtest_utilization (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
90342025-04-30T06:44:08.523ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
90352025-04-30T06:44:08.523ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
90362025-04-30T06:44:08.523ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
90372025-04-30T06:44:08.526ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
90382025-04-30T06:44:08.533ZDEBGtest_utilization (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet result = Ok(())
90392025-04-30T06:44:08.533ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
90402025-04-30T06:44:08.534ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
90412025-04-30T06:44:08.534ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
90422025-04-30T06:44:08.614ZDEBGtest_utilization (BackgroundTasks): begin collection from Sled Agent background_task = inventory_collection sled_agent_url = http://[::1]:39365
90432025-04-30T06:44:08.614ZDEBGtest_utilization (BackgroundTasks): client request SledAgent = http://[::1]:39365 background_task = inventory_collection body = None method = GET uri = http://[::1]:39365/inventory
90442025-04-30T06:44:08.615ZTRACtest_utilization (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory
90452025-04-30T06:44:08.615ZTRACtest_utilization (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
90462025-04-30T06:44:08.616ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
90472025-04-30T06:44:08.616ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
90482025-04-30T06:44:08.627ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
90492025-04-30T06:44:08.627ZINFOtest_utilization (dropshot): accepted connection local_addr = [::1]:39365 remote_addr = [::1]:45579 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
90502025-04-30T06:44:08.636ZTRACtest_utilization (dropshot): incoming request local_addr = [::1]:39365 method = GET remote_addr = [::1]:45579 req_id = 02da05ba-e23d-424a-be24-f682311dcfb5 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /inventory
90512025-04-30T06:44:08.637ZINFOtest_utilization (dropshot): request completed latency_us = 671 local_addr = [::1]:39365 method = GET remote_addr = [::1]:45579 req_id = 02da05ba-e23d-424a-be24-f682311dcfb5 response_code = 200 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /inventory
90522025-04-30T06:44:08.639ZDEBGtest_utilization (BackgroundTasks): client response SledAgent = http://[::1]:39365 background_task = inventory_collection result = Ok(Response { url: "http://[::1]:39365/inventory", status: 200, headers: {"content-type": "application/json", "x-request-id": "02da05ba-e23d-424a-be24-f682311dcfb5", "content-length": "3141", "date": "Wed, 30 Apr 2025 06:44:07 GMT"} })
90532025-04-30T06:44:08.641ZDEBGtest_utilization (BackgroundTasks): begin collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
90542025-04-30T06:44:08.642ZDEBGtest_utilization (BackgroundTasks): end collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
90552025-04-30T06:44:08.642ZDEBGtest_utilization (BackgroundTasks): finished collection background_task = inventory_collection
90562025-04-30T06:44:08.646ZTRACtest_utilization (BackgroundTasks): authorize begin action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory
90572025-04-30T06:44:08.647ZTRACtest_utilization (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
90582025-04-30T06:44:08.647ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
90592025-04-30T06:44:08.647ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
90602025-04-30T06:44:08.648ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
90612025-04-30T06:44:08.651ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
90622025-04-30T06:44:08.661ZDEBGtest_utilization (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory result = Ok(())
90632025-04-30T06:44:08.661ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
90642025-04-30T06:44:08.661ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
90652025-04-30T06:44:08.662ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
90662025-04-30T06:44:08.672ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
90672025-04-30T06:44:08.677ZDEBGtest_utilization (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory result = Ok(())
90682025-04-30T06:44:08.678ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
90692025-04-30T06:44:08.678ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
90702025-04-30T06:44:08.679ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
90712025-04-30T06:44:08.697ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
90722025-04-30T06:44:08.698ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
90732025-04-30T06:44:08.698ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
90742025-04-30T06:44:08.784ZWARNtest_utilization (datastore): Non-retryable transaction failure err = DatabaseError(UniqueViolation, "duplicate key value violates unique constraint \\"inv_caboose_pkey\\"") transaction = inventory_insert_collection
90752025-04-30T06:44:08.784ZWARNtest_utilization (BackgroundTasks): inventory collection failed background_task = inventory_collection error = failed to collect inventory: saving inventory to database: Internal Error: unexpected database error: duplicate key value violates unique constraint "inv_caboose_pkey"
90762025-04-30T06:44:08.784ZDEBGtest_utilization (BackgroundTasks): activation complete background_task = inventory_collection elapsed = 578.180958ms iteration = 17 status = {"error":"failed to collect inventory: saving inventory to database: Internal Error: unexpected database error: duplicate key value violates unique constraint \\"inv_caboose_pkey\\""}
90772025-04-30T06:44:08.784ZDEBGtest_utilization (BackgroundTasks): activating background_task = inventory_collection iteration = 18 reason = Signaled
90782025-04-30T06:44:08.784ZTRACtest_utilization (BackgroundTasks): authorize begin action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory
90792025-04-30T06:44:08.784ZTRACtest_utilization (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
90802025-04-30T06:44:08.784ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
90812025-04-30T06:44:08.784ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
90822025-04-30T06:44:08.786ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
90832025-04-30T06:44:08.788ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
90842025-04-30T06:44:08.795ZDEBGtest_utilization (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory result = Ok(())
90852025-04-30T06:44:08.795ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
90862025-04-30T06:44:08.795ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
90872025-04-30T06:44:08.795ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
90882025-04-30T06:44:08.799ZDEBGtest_utilization: attempt to fetch latest inventory collection result = Ok("found")
90892025-04-30T06:44:08.804ZDEBGtest_utilization (BackgroundTasks): inventory_prune_one: nothing eligible for removal (too few) background_task = inventory_collection candidates = [(13d8b8d9-6c6b-47a6-9190-952544edb75b, 0)]
90902025-04-30T06:44:08.804ZTRACtest_utilization (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal
90912025-04-30T06:44:08.804ZTRACtest_utilization (dns): buffer buffer = Length: 67 (0x43) bytes\n0000: 66 1f 01 00 00 01 00 00 00 00 00 01 04 5f 6d 67 f............_mg\n0010: 73 04 5f 74 63 70 0d 63 6f 6e 74 72 6f 6c 2d 70 s._tcp.control-p\n0020: 6c 61 6e 65 05 6f 78 69 64 65 08 69 6e 74 65 72 lane.oxide.inter\n0030: 6e 61 6c 00 00 21 00 01 00 00 29 04 d0 00 00 00 nal..!....).....\n0040: 00 00 00 ... kind = dns peer_addr = [::1]:60016 req_id = ba874a6a-5f73-4e86-996b-b273967e3cb9
90922025-04-30T06:44:08.804ZDEBGtest_utilization (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 26143,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_mgs._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_mgs._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 4,\n 95,\n 109,\n 103,\n 115,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:60016 req_id = ba874a6a-5f73-4e86-996b-b273967e3cb9
90932025-04-30T06:44:08.806ZDEBGtest_utilization (store): query key key = _mgs._tcp
90942025-04-30T06:44:08.807ZDEBGtest_utilization (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
90952025-04-30T06:44:08.807ZDEBGtest_utilization (dns): dns response additional_records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal"), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] kind = dns peer_addr = [::1]:60016 query = LowerQuery { name: LowerName(Name("_mgs._tcp.control-plane.oxide.internal.")), original: Query { name: Name("_mgs._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN } } records = [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 48759, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal") })) }] req_id = ba874a6a-5f73-4e86-996b-b273967e3cb9
90962025-04-30T06:44:08.807ZDEBGtest_utilization (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgs._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 48759, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }], valid_until: Instant { tv_sec: 4313, tv_nsec: 606387722 } })
90972025-04-30T06:44:08.807ZTRACtest_utilization (DnsResolver): lookup_service_targets: looking up SRV target
90982025-04-30T06:44:08.807ZTRACtest_utilization (dns): buffer buffer = Length: 108 (0x6c) bytes\n0000: fa 87 01 00 00 01 00 00 00 00 00 01 2d 64 65 6e ............-den\n0010: 64 72 69 74 65 2d 62 36 64 36 35 33 34 31 2d 31 drite-b6d65341-1\n0020: 36 37 63 2d 34 31 64 66 2d 39 62 35 63 2d 34 31 67c-41df-9b5c-41\n0030: 63 64 65 64 39 39 63 32 32 39 04 68 6f 73 74 0d cded99c229.host.\n0040: 63 6f 6e 74 72 6f 6c 2d 70 6c 61 6e 65 05 6f 78 control-plane.ox\n0050: 69 64 65 08 69 6e 74 65 72 6e 61 6c 00 00 1c 00 ide.internal....\n0060: 01 00 00 29 04 d0 00 00 00 00 00 00 ...)........ kind = dns peer_addr = [::1]:57917 req_id = a1af158f-8292-47db-b29f-6fd0302de506
90992025-04-30T06:44:08.807ZDEBGtest_utilization (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 64135,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n query_type: AAAA,\n query_class: IN,\n },\n },\n original: [\n 45,\n 100,\n 101,\n 110,\n 100,\n 114,\n 105,\n 116,\n 101,\n 45,\n 98,\n 54,\n 100,\n 54,\n 53,\n 51,\n 52,\n 49,\n 45,\n 49,\n 54,\n 55,\n 99,\n 45,\n 52,\n 49,\n 100,\n 102,\n 45,\n 57,\n 98,\n 53,\n 99,\n 45,\n 52,\n 49,\n 99,\n 100,\n 101,\n 100,\n 57,\n 57,\n 99,\n 50,\n 50,\n 57,\n 4,\n 104,\n 111,\n 115,\n 116,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 28,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:57917 req_id = a1af158f-8292-47db-b29f-6fd0302de506
91002025-04-30T06:44:08.808ZDEBGtest_utilization (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
91012025-04-30T06:44:08.808ZDEBGtest_utilization (dns): dns response additional_records = [] kind = dns peer_addr = [::1]:57917 query = LowerQuery { name: LowerName(Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.")), original: Query { name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), query_type: AAAA, query_class: IN } } records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] req_id = a1af158f-8292-47db-b29f-6fd0302de506
91022025-04-30T06:44:08.886ZTRACtest_utilization (DnsResolver): lookup_all_socket_v6 srv dns_name = _clickhouse-admin-keeper._tcp.control-plane.oxide.internal
91032025-04-30T06:44:08.887ZTRACtest_utilization (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory
91042025-04-30T06:44:08.888ZTRACtest_utilization (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
91052025-04-30T06:44:08.888ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
91062025-04-30T06:44:08.888ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
91072025-04-30T06:44:08.889ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
91082025-04-30T06:44:08.889ZTRACtest_utilization (dns): buffer buffer = Length: 87 (0x57) bytes\n0000: 08 4a 01 00 00 01 00 00 00 00 00 01 18 5f 63 6c .J..........._cl\n0010: 69 63 6b 68 6f 75 73 65 2d 61 64 6d 69 6e 2d 6b ickhouse-admin-k\n0020: 65 65 70 65 72 04 5f 74 63 70 0d 63 6f 6e 74 72 eeper._tcp.contr\n0030: 6f 6c 2d 70 6c 61 6e 65 05 6f 78 69 64 65 08 69 ol-plane.oxide.i\n0040: 6e 74 65 72 6e 61 6c 00 00 21 00 01 00 00 29 04 nternal..!....).\n0050: d0 00 00 00 00 00 00 ....... kind = dns peer_addr = [::1]:63808 req_id = 46ce2023-6216-4f54-96c3-8fbef7e99ef8
91092025-04-30T06:44:08.889ZDEBGtest_utilization (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 2122,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 24,\n 95,\n 99,\n 108,\n 105,\n 99,\n 107,\n 104,\n 111,\n 117,\n 115,\n 101,\n 45,\n 97,\n 100,\n 109,\n 105,\n 110,\n 45,\n 107,\n 101,\n 101,\n 112,\n 101,\n 114,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:63808 req_id = 46ce2023-6216-4f54-96c3-8fbef7e99ef8
91102025-04-30T06:44:08.890ZDEBGtest_utilization (store): query key key = _clickhouse-admin-keeper._tcp
91112025-04-30T06:44:08.890ZERROtest_utilization (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 2122, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 1, }, query: WireQuery { query: LowerQuery { name: LowerName( Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), ), original: Query { name: Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN, }, }, original: [ 24, 95, 99, 108, 105, 99, 107, 104, 111, 117, 115, 101, 45, 97, 100, 109, 105, 110, 45, 107, 101, 101, 112, 101, 114, 4, 95, 116, 99, 112, 13, 99, 111, 110, 116, 114, 111, 108, 45, 112, 108, 97, 110, 101, 5, 111, 120, 105, 100, 101, 8, 105, 110, 116, 101, 114, 110, 97, 108, 0, 0, 33, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: Some( Edns { rcode_high: 0, version: 0, dnssec_ok: false, max_payload: 1232, options: OPT { options: {}, }, }, ), } NXDOMAIN: no records found for name: "_clickhouse-admin-keeper._tcp.control-plane.oxide.internal." kind = dns peer_addr = [::1]:63808 req_id = 46ce2023-6216-4f54-96c3-8fbef7e99ef8
91122025-04-30T06:44:08.891ZDEBGtest_utilization (BackgroundTasks): begin collection background_task = inventory_collection
91132025-04-30T06:44:08.891ZDEBGtest_utilization (BackgroundTasks): begin collection from MGS background_task = inventory_collection mgs_url = http://[::1]:48759
91142025-04-30T06:44:08.891ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/ignition
91152025-04-30T06:44:08.891ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
91162025-04-30T06:44:08.901ZDEBGtest_utilization (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory result = Ok(())
91172025-04-30T06:44:08.901ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
91182025-04-30T06:44:08.901ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
91192025-04-30T06:44:08.904ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
91202025-04-30T06:44:08.909ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
91212025-04-30T06:44:08.909ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
91222025-04-30T06:44:08.912ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
91232025-04-30T06:44:08.914ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/ignition", status: 200, headers: {"content-type": "application/json", "x-request-id": "eacb25a0-6aa8-4e64-85dc-f58b1dde501c", "content-length": "835", "date": "Wed, 30 Apr 2025 06:44:08 GMT"} })
91242025-04-30T06:44:08.914ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0
91252025-04-30T06:44:08.922ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "9ccf5821-ce6f-4fb0-a04f-2f94028a1110", "content-length": "734", "date": "Wed, 30 Apr 2025 06:44:08 GMT"} })
91262025-04-30T06:44:08.922ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0/component/sp/caboose?firmware_slot=0
91272025-04-30T06:44:08.922ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "e023a17a-7e12-41f6-9d74-2ea20564e0f8", "content-length": "111", "date": "Wed, 30 Apr 2025 06:44:08 GMT"} })
91282025-04-30T06:44:08.922ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0/component/sp/caboose?firmware_slot=1
91292025-04-30T06:44:08.922ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "4238d41f-07c7-4c9d-8d29-fa7374d5fb09", "content-length": "111", "date": "Wed, 30 Apr 2025 06:44:08 GMT"} })
91302025-04-30T06:44:08.922ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0/component/rot/caboose?firmware_slot=0
91312025-04-30T06:44:08.927ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "181d4f73-46af-4227-9008-ec4043fa6665", "content-length": "170", "date": "Wed, 30 Apr 2025 06:44:08 GMT"} })
91322025-04-30T06:44:08.927ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0/component/rot/caboose?firmware_slot=1
91332025-04-30T06:44:08.927ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "0ea97c3a-0006-4f0d-85b5-ea203a18f1a7", "content-length": "170", "date": "Wed, 30 Apr 2025 06:44:08 GMT"} })
91342025-04-30T06:44:08.927ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0/component/stage0/caboose?firmware_slot=0
91352025-04-30T06:44:08.930ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "b2e145fa-c066-40e6-a2ac-01950283e9b5", "content-length": "179", "date": "Wed, 30 Apr 2025 06:44:08 GMT"} })
91362025-04-30T06:44:08.930ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0/component/stage0/caboose?firmware_slot=1
91372025-04-30T06:44:08.932ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "6f9907c1-bcf9-4916-ad02-b5edffcb4f4b", "content-length": "179", "date": "Wed, 30 Apr 2025 06:44:08 GMT"} })
91382025-04-30T06:44:08.933ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0/component/rot/cmpa
91392025-04-30T06:44:08.935ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "21f22d48-1caa-4d03-aec4-0b77c5ff99d1", "content-length": "702", "date": "Wed, 30 Apr 2025 06:44:08 GMT"} })
91402025-04-30T06:44:08.935ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0/component/rot/cfpa
91412025-04-30T06:44:08.940ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "8455777c-3cb6-4455-9510-1ba8ea6602f7", "content-length": "727", "date": "Wed, 30 Apr 2025 06:44:08 GMT"} })
91422025-04-30T06:44:08.941ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0/component/rot/cfpa
91432025-04-30T06:44:08.944ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "b1f62e16-96e7-45aa-b5cd-b15760b96849", "content-length": "729", "date": "Wed, 30 Apr 2025 06:44:08 GMT"} })
91442025-04-30T06:44:08.944ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0/component/rot/cfpa
91452025-04-30T06:44:08.950ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "30b0eb48-0601-46b4-ad84-081dfec0b242", "content-length": "728", "date": "Wed, 30 Apr 2025 06:44:08 GMT"} })
91462025-04-30T06:44:08.950ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1
91472025-04-30T06:44:08.953ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1", status: 200, headers: {"content-type": "application/json", "x-request-id": "132d8993-c3ef-42a5-b6ec-6e270f61baaf", "content-length": "734", "date": "Wed, 30 Apr 2025 06:44:08 GMT"} })
91482025-04-30T06:44:08.954ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1/component/sp/caboose?firmware_slot=0
91492025-04-30T06:44:08.958ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "bb70704c-89f2-4adf-8f23-99668b3e7269", "content-length": "111", "date": "Wed, 30 Apr 2025 06:44:08 GMT"} })
91502025-04-30T06:44:08.967ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1/component/sp/caboose?firmware_slot=1
91512025-04-30T06:44:08.973ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "840e4d87-f995-4752-b451-3792afe385b3", "content-length": "111", "date": "Wed, 30 Apr 2025 06:44:08 GMT"} })
91522025-04-30T06:44:08.974ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1/component/rot/caboose?firmware_slot=0
91532025-04-30T06:44:08.981ZDEBGtest_utilization: attempt to fetch latest inventory collection result = Ok("found")
91542025-04-30T06:44:08.983ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "f641b755-0394-45ad-8406-f474666391b1", "content-length": "170", "date": "Wed, 30 Apr 2025 06:44:08 GMT"} })
91552025-04-30T06:44:08.983ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1/component/rot/caboose?firmware_slot=1
91562025-04-30T06:44:08.985ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "23dc590f-8d0e-489d-92d0-847364199432", "content-length": "170", "date": "Wed, 30 Apr 2025 06:44:08 GMT"} })
91572025-04-30T06:44:08.985ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1/component/stage0/caboose?firmware_slot=0
91582025-04-30T06:44:08.988ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "963414ce-4b77-48d3-8523-22c8f0d9298e", "content-length": "179", "date": "Wed, 30 Apr 2025 06:44:08 GMT"} })
91592025-04-30T06:44:08.989ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1/component/stage0/caboose?firmware_slot=1
91602025-04-30T06:44:08.991ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "af562af4-31a0-42fe-bb50-aaa1b260927e", "content-length": "179", "date": "Wed, 30 Apr 2025 06:44:08 GMT"} })
91612025-04-30T06:44:08.991ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1/component/rot/cmpa
91622025-04-30T06:44:08.993ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "2b6d5b7a-dc39-4d7d-92f1-b4190d4c1378", "content-length": "702", "date": "Wed, 30 Apr 2025 06:44:08 GMT"} })
91632025-04-30T06:44:08.993ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1/component/rot/cfpa
91642025-04-30T06:44:08.994ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "529ba24b-07ff-4e2e-a634-f27a7f2eac5b", "content-length": "727", "date": "Wed, 30 Apr 2025 06:44:08 GMT"} })
91652025-04-30T06:44:08.994ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1/component/rot/cfpa
91662025-04-30T06:44:08.996ZDEBGtest_utilization (collection-task): sent timer-based collection request to the collection task collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
91672025-04-30T06:44:08.996ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "b64642db-440a-485e-a205-70b8c3bc00cd", "content-length": "729", "date": "Wed, 30 Apr 2025 06:44:08 GMT"} })
91682025-04-30T06:44:08.996ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1/component/rot/cfpa
91692025-04-30T06:44:08.996ZDEBGtest_utilization (collection-task): collecting from producer collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
91702025-04-30T06:44:08.998ZTRACtest_utilization (collection-task): sent collection request to producer collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
91712025-04-30T06:44:08.998ZDEBGtest_utilization (collection-task): collected results from producer collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 n_results = 1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
91722025-04-30T06:44:08.998ZTRACtest_utilization (collection-task): forwarded results to main collection loop collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
91732025-04-30T06:44:08.998ZTRACtest_utilization (collection-task): top of inner collection loop, waiting for next request collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
91742025-04-30T06:44:08.999ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "92aaa1bc-4505-4c97-9511-1559671d0030", "content-length": "728", "date": "Wed, 30 Apr 2025 06:44:08 GMT"} })
91752025-04-30T06:44:09.000ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0
91762025-04-30T06:44:09.002ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "8893363f-c932-4447-9069-36cc8e838791", "content-length": "723", "date": "Wed, 30 Apr 2025 06:44:08 GMT"} })
91772025-04-30T06:44:09.003ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0/component/sp/caboose?firmware_slot=0
91782025-04-30T06:44:09.006ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "119d4ce9-8402-4e2c-9214-ee1776568f7c", "content-length": "109", "date": "Wed, 30 Apr 2025 06:44:08 GMT"} })
91792025-04-30T06:44:09.006ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0/component/sp/caboose?firmware_slot=1
91802025-04-30T06:44:09.008ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "305e137c-eb00-44e3-aba0-3332bfd42cd9", "content-length": "109", "date": "Wed, 30 Apr 2025 06:44:08 GMT"} })
91812025-04-30T06:44:09.009ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0/component/rot/caboose?firmware_slot=0
91822025-04-30T06:44:09.012ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "cc5f6452-404f-4efa-aa5f-75f55e8419bb", "content-length": "169", "date": "Wed, 30 Apr 2025 06:44:08 GMT"} })
91832025-04-30T06:44:09.012ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0/component/rot/caboose?firmware_slot=1
91842025-04-30T06:44:09.015ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "9617b653-245c-43e9-8752-efcbf65e6742", "content-length": "169", "date": "Wed, 30 Apr 2025 06:44:08 GMT"} })
91852025-04-30T06:44:09.015ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0/component/stage0/caboose?firmware_slot=0
91862025-04-30T06:44:09.020ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "a673a152-de17-4e16-9486-62cb715ca869", "content-length": "178", "date": "Wed, 30 Apr 2025 06:44:08 GMT"} })
91872025-04-30T06:44:09.020ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0/component/stage0/caboose?firmware_slot=1
91882025-04-30T06:44:09.024ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "dbca4899-9b43-4689-97a0-684efa306821", "content-length": "178", "date": "Wed, 30 Apr 2025 06:44:08 GMT"} })
91892025-04-30T06:44:09.024ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0/component/rot/cmpa
91902025-04-30T06:44:09.025ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "3eb75d0e-f47a-4a29-814c-15bbf02e9b04", "content-length": "702", "date": "Wed, 30 Apr 2025 06:44:08 GMT"} })
91912025-04-30T06:44:09.025ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0/component/rot/cfpa
91922025-04-30T06:44:09.026ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "90f2e36b-82a4-4e6d-890d-de49942767a3", "content-length": "727", "date": "Wed, 30 Apr 2025 06:44:08 GMT"} })
91932025-04-30T06:44:09.027ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0/component/rot/cfpa
91942025-04-30T06:44:09.028ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "c9c4a96d-52dd-4b41-9327-449fbe10f1bd", "content-length": "729", "date": "Wed, 30 Apr 2025 06:44:08 GMT"} })
91952025-04-30T06:44:09.028ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0/component/rot/cfpa
91962025-04-30T06:44:09.029ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "843c6bcd-5e5f-4b66-b854-469b8811d82d", "content-length": "728", "date": "Wed, 30 Apr 2025 06:44:08 GMT"} })
91972025-04-30T06:44:09.029ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1
91982025-04-30T06:44:09.031ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1", status: 200, headers: {"content-type": "application/json", "x-request-id": "f4900d06-a9f7-48fb-bbc2-de8b63244fb9", "content-length": "723", "date": "Wed, 30 Apr 2025 06:44:08 GMT"} })
91992025-04-30T06:44:09.031ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1/component/sp/caboose?firmware_slot=0
92002025-04-30T06:44:09.031ZTRACtest_utilization (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory
92012025-04-30T06:44:09.031ZTRACtest_utilization (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
92022025-04-30T06:44:09.031ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
92032025-04-30T06:44:09.031ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
92042025-04-30T06:44:09.033ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
92052025-04-30T06:44:09.035ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "a2edff9d-6a7e-464a-b895-7b1caf0a9cdc", "content-length": "109", "date": "Wed, 30 Apr 2025 06:44:08 GMT"} })
92062025-04-30T06:44:09.035ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1/component/sp/caboose?firmware_slot=1
92072025-04-30T06:44:09.037ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
92082025-04-30T06:44:09.044ZDEBGtest_utilization (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory result = Ok(())
92092025-04-30T06:44:09.045ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
92102025-04-30T06:44:09.045ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
92112025-04-30T06:44:09.045ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
92122025-04-30T06:44:09.046ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "af0f81ac-3ad7-491d-842f-0b5a9dbe2e04", "content-length": "109", "date": "Wed, 30 Apr 2025 06:44:08 GMT"} })
92132025-04-30T06:44:09.046ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1/component/rot/caboose?firmware_slot=0
92142025-04-30T06:44:09.048ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "5cdddf72-c233-42b3-8dd5-f8df636ec5b7", "content-length": "169", "date": "Wed, 30 Apr 2025 06:44:08 GMT"} })
92152025-04-30T06:44:09.048ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1/component/rot/caboose?firmware_slot=1
92162025-04-30T06:44:09.049ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
92172025-04-30T06:44:09.049ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
92182025-04-30T06:44:09.050ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
92192025-04-30T06:44:09.053ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "3b9f6b62-3538-4738-9ec4-347eb0a7fa4f", "content-length": "169", "date": "Wed, 30 Apr 2025 06:44:08 GMT"} })
92202025-04-30T06:44:09.053ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1/component/stage0/caboose?firmware_slot=0
92212025-04-30T06:44:09.055ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "49ce6994-06a6-42d0-965b-1e121675d904", "content-length": "178", "date": "Wed, 30 Apr 2025 06:44:08 GMT"} })
92222025-04-30T06:44:09.055ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1/component/stage0/caboose?firmware_slot=1
92232025-04-30T06:44:09.058ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "5615ace0-f897-4a85-8240-52728fc19aea", "content-length": "178", "date": "Wed, 30 Apr 2025 06:44:08 GMT"} })
92242025-04-30T06:44:09.058ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1/component/rot/cmpa
92252025-04-30T06:44:09.059ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "520795fd-b8b6-44a9-96a6-d01205cde4de", "content-length": "702", "date": "Wed, 30 Apr 2025 06:44:08 GMT"} })
92262025-04-30T06:44:09.059ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1/component/rot/cfpa
92272025-04-30T06:44:09.062ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "c06cd844-eefb-4edc-9a65-765d5d25cce0", "content-length": "727", "date": "Wed, 30 Apr 2025 06:44:08 GMT"} })
92282025-04-30T06:44:09.062ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1/component/rot/cfpa
92292025-04-30T06:44:09.063ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "14a1dd45-4688-4621-9c34-906a1fa0b12e", "content-length": "729", "date": "Wed, 30 Apr 2025 06:44:08 GMT"} })
92302025-04-30T06:44:09.063ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1/component/rot/cfpa
92312025-04-30T06:44:09.066ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "e44259c2-280e-49b0-bc55-a0567d0b0bf4", "content-length": "728", "date": "Wed, 30 Apr 2025 06:44:08 GMT"} })
92322025-04-30T06:44:09.066ZTRACtest_utilization (BackgroundTasks): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet
92332025-04-30T06:44:09.066ZTRACtest_utilization (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
92342025-04-30T06:44:09.066ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
92352025-04-30T06:44:09.066ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
92362025-04-30T06:44:09.067ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
92372025-04-30T06:44:09.078ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
92382025-04-30T06:44:09.090ZDEBGtest_utilization (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet result = Ok(())
92392025-04-30T06:44:09.090ZTRACtest_utilization (BackgroundTasks): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet
92402025-04-30T06:44:09.090ZTRACtest_utilization (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
92412025-04-30T06:44:09.090ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
92422025-04-30T06:44:09.091ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
92432025-04-30T06:44:09.091ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
92442025-04-30T06:44:09.094ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
92452025-04-30T06:44:09.100ZDEBGtest_utilization (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet result = Ok(())
92462025-04-30T06:44:09.101ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
92472025-04-30T06:44:09.101ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
92482025-04-30T06:44:09.102ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
92492025-04-30T06:44:09.208ZDEBGtest_utilization (BackgroundTasks): begin collection from Sled Agent background_task = inventory_collection sled_agent_url = http://[::1]:39365
92502025-04-30T06:44:09.208ZDEBGtest_utilization (BackgroundTasks): client request SledAgent = http://[::1]:39365 background_task = inventory_collection body = None method = GET uri = http://[::1]:39365/inventory
92512025-04-30T06:44:09.209ZINFOtest_utilization (dropshot): accepted connection local_addr = [::1]:39365 remote_addr = [::1]:55597 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
92522025-04-30T06:44:09.210ZTRACtest_utilization (dropshot): incoming request local_addr = [::1]:39365 method = GET remote_addr = [::1]:55597 req_id = 16abedb2-1845-49b6-af8d-267f0fb2172c sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /inventory
92532025-04-30T06:44:09.211ZINFOtest_utilization (dropshot): request completed latency_us = 661 local_addr = [::1]:39365 method = GET remote_addr = [::1]:55597 req_id = 16abedb2-1845-49b6-af8d-267f0fb2172c response_code = 200 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /inventory
92542025-04-30T06:44:09.212ZDEBGtest_utilization (BackgroundTasks): client response SledAgent = http://[::1]:39365 background_task = inventory_collection result = Ok(Response { url: "http://[::1]:39365/inventory", status: 200, headers: {"content-type": "application/json", "x-request-id": "16abedb2-1845-49b6-af8d-267f0fb2172c", "content-length": "3141", "date": "Wed, 30 Apr 2025 06:44:08 GMT"} })
92552025-04-30T06:44:09.215ZDEBGtest_utilization (BackgroundTasks): begin collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
92562025-04-30T06:44:09.215ZDEBGtest_utilization (BackgroundTasks): end collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
92572025-04-30T06:44:09.215ZDEBGtest_utilization (BackgroundTasks): finished collection background_task = inventory_collection
92582025-04-30T06:44:09.217ZTRACtest_utilization (BackgroundTasks): authorize begin action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory
92592025-04-30T06:44:09.217ZTRACtest_utilization (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
92602025-04-30T06:44:09.217ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
92612025-04-30T06:44:09.217ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
92622025-04-30T06:44:09.217ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
92632025-04-30T06:44:09.224ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
92642025-04-30T06:44:09.230ZDEBGtest_utilization (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory result = Ok(())
92652025-04-30T06:44:09.231ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
92662025-04-30T06:44:09.231ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
92672025-04-30T06:44:09.232ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
92682025-04-30T06:44:09.254ZDEBGtest_utilization: attempt to fetch latest inventory collection result = Ok("found")
92692025-04-30T06:44:09.304ZWARNtest_utilization (datastore): Non-retryable transaction failure err = DatabaseError(UniqueViolation, "duplicate key value violates unique constraint \\"inv_caboose_pkey\\"") transaction = inventory_insert_collection
92702025-04-30T06:44:09.306ZWARNtest_utilization (BackgroundTasks): inventory collection failed background_task = inventory_collection error = failed to collect inventory: saving inventory to database: Internal Error: unexpected database error: duplicate key value violates unique constraint "inv_caboose_pkey"
92712025-04-30T06:44:09.306ZDEBGtest_utilization (BackgroundTasks): activation complete background_task = inventory_collection elapsed = 521.291048ms iteration = 18 status = {"error":"failed to collect inventory: saving inventory to database: Internal Error: unexpected database error: duplicate key value violates unique constraint \\"inv_caboose_pkey\\""}
92722025-04-30T06:44:09.306ZDEBGtest_utilization (BackgroundTasks): activating background_task = inventory_collection iteration = 19 reason = Signaled
92732025-04-30T06:44:09.306ZTRACtest_utilization (BackgroundTasks): authorize begin action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory
92742025-04-30T06:44:09.306ZTRACtest_utilization (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
92752025-04-30T06:44:09.306ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
92762025-04-30T06:44:09.306ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
92772025-04-30T06:44:09.307ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
92782025-04-30T06:44:09.307ZTRACtest_utilization (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory
92792025-04-30T06:44:09.307ZTRACtest_utilization (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
92802025-04-30T06:44:09.307ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
92812025-04-30T06:44:09.307ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
92822025-04-30T06:44:09.310ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
92832025-04-30T06:44:09.314ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
92842025-04-30T06:44:09.319ZDEBGtest_utilization (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory result = Ok(())
92852025-04-30T06:44:09.319ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
92862025-04-30T06:44:09.319ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
92872025-04-30T06:44:09.320ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
92882025-04-30T06:44:09.320ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
92892025-04-30T06:44:09.326ZDEBGtest_utilization (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory result = Ok(())
92902025-04-30T06:44:09.360ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
92912025-04-30T06:44:09.360ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
92922025-04-30T06:44:09.361ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
92932025-04-30T06:44:09.362ZDEBGtest_utilization (results-sink): inserting 1 samples into database collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
92942025-04-30T06:44:09.362ZDEBGtest_utilization (results-sink-cluster): inserting 1 samples into database collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
92952025-04-30T06:44:09.362ZDEBGtest_utilization (clickhouse-client): unrolling 1 total samples collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 4d1485f0-110e-4161-9a33-c4610d68be5d
92962025-04-30T06:44:09.363ZTRACtest_utilization (clickhouse-client): inserting data collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 4d1485f0-110e-4161-9a33-c4610d68be5d n_columns = 4 n_rows = 2 sql = INSERT INTO oximeter.fields_string FORMAT Native
92972025-04-30T06:44:09.369ZDEBGtest_utilization (clickhouse-client): inserted rows into table collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 4d1485f0-110e-4161-9a33-c4610d68be5d n_rows = 2 table_name = fields_string
92982025-04-30T06:44:09.373ZTRACtest_utilization (clickhouse-client): inserting data collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 4d1485f0-110e-4161-9a33-c4610d68be5d n_columns = 4 n_rows = 1 sql = INSERT INTO oximeter.measurements_i64 FORMAT Native
92992025-04-30T06:44:09.378ZDEBGtest_utilization (BackgroundTasks): inventory_prune_one: nothing eligible for removal (too few) background_task = inventory_collection candidates = [(13d8b8d9-6c6b-47a6-9190-952544edb75b, 0)]
93002025-04-30T06:44:09.378ZTRACtest_utilization (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal
93012025-04-30T06:44:09.380ZDEBGtest_utilization (clickhouse-client): inserted rows into table collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 4d1485f0-110e-4161-9a33-c4610d68be5d n_rows = 1 table_name = measurements_i64
93022025-04-30T06:44:09.382ZTRACtest_utilization (results-sink): successfully inserted samples collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
93032025-04-30T06:44:09.386ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
93042025-04-30T06:44:09.386ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
93052025-04-30T06:44:09.390ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
93062025-04-30T06:44:09.390ZTRACtest_utilization (dns): buffer buffer = Length: 67 (0x43) bytes\n0000: cc 5e 01 00 00 01 00 00 00 00 00 01 04 5f 6d 67 .^..........._mg\n0010: 73 04 5f 74 63 70 0d 63 6f 6e 74 72 6f 6c 2d 70 s._tcp.control-p\n0020: 6c 61 6e 65 05 6f 78 69 64 65 08 69 6e 74 65 72 lane.oxide.inter\n0030: 6e 61 6c 00 00 21 00 01 00 00 29 04 d0 00 00 00 nal..!....).....\n0040: 00 00 00 ... kind = dns peer_addr = [::1]:60260 req_id = 425e2aa7-0029-46bc-a9f6-924155ff5f62
93072025-04-30T06:44:09.391ZDEBGtest_utilization (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 52318,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_mgs._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_mgs._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 4,\n 95,\n 109,\n 103,\n 115,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:60260 req_id = 425e2aa7-0029-46bc-a9f6-924155ff5f62
93082025-04-30T06:44:09.393ZDEBGtest_utilization (store): query key key = _mgs._tcp
93092025-04-30T06:44:09.395ZDEBGtest_utilization (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
93102025-04-30T06:44:09.395ZDEBGtest_utilization (dns): dns response additional_records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal"), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] kind = dns peer_addr = [::1]:60260 query = LowerQuery { name: LowerName(Name("_mgs._tcp.control-plane.oxide.internal.")), original: Query { name: Name("_mgs._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN } } records = [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 48759, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal") })) }] req_id = 425e2aa7-0029-46bc-a9f6-924155ff5f62
93112025-04-30T06:44:09.395ZDEBGtest_utilization (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgs._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 48759, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }], valid_until: Instant { tv_sec: 4314, tv_nsec: 194504927 } })
93122025-04-30T06:44:09.396ZTRACtest_utilization (DnsResolver): lookup_service_targets: looking up SRV target
93132025-04-30T06:44:09.398ZTRACtest_utilization (dns): buffer buffer = Length: 108 (0x6c) bytes\n0000: 4c f5 01 00 00 01 00 00 00 00 00 01 2d 64 65 6e L...........-den\n0010: 64 72 69 74 65 2d 62 36 64 36 35 33 34 31 2d 31 drite-b6d65341-1\n0020: 36 37 63 2d 34 31 64 66 2d 39 62 35 63 2d 34 31 67c-41df-9b5c-41\n0030: 63 64 65 64 39 39 63 32 32 39 04 68 6f 73 74 0d cded99c229.host.\n0040: 63 6f 6e 74 72 6f 6c 2d 70 6c 61 6e 65 05 6f 78 control-plane.ox\n0050: 69 64 65 08 69 6e 74 65 72 6e 61 6c 00 00 1c 00 ide.internal....\n0060: 01 00 00 29 04 d0 00 00 00 00 00 00 ...)........ kind = dns peer_addr = [::1]:52074 req_id = 261e2618-e413-43a4-b85c-603c6865d7ce
93142025-04-30T06:44:09.400ZDEBGtest_utilization (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 19701,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n query_type: AAAA,\n query_class: IN,\n },\n },\n original: [\n 45,\n 100,\n 101,\n 110,\n 100,\n 114,\n 105,\n 116,\n 101,\n 45,\n 98,\n 54,\n 100,\n 54,\n 53,\n 51,\n 52,\n 49,\n 45,\n 49,\n 54,\n 55,\n 99,\n 45,\n 52,\n 49,\n 100,\n 102,\n 45,\n 57,\n 98,\n 53,\n 99,\n 45,\n 52,\n 49,\n 99,\n 100,\n 101,\n 100,\n 57,\n 57,\n 99,\n 50,\n 50,\n 57,\n 4,\n 104,\n 111,\n 115,\n 116,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 28,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:52074 req_id = 261e2618-e413-43a4-b85c-603c6865d7ce
93152025-04-30T06:44:09.400ZDEBGtest_utilization (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
93162025-04-30T06:44:09.400ZDEBGtest_utilization (dns): dns response additional_records = [] kind = dns peer_addr = [::1]:52074 query = LowerQuery { name: LowerName(Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.")), original: Query { name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), query_type: AAAA, query_class: IN } } records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] req_id = 261e2618-e413-43a4-b85c-603c6865d7ce
93172025-04-30T06:44:09.487ZTRACtest_utilization (DnsResolver): lookup_all_socket_v6 srv dns_name = _clickhouse-admin-keeper._tcp.control-plane.oxide.internal
93182025-04-30T06:44:09.488ZWARNtest_utilization (results-sink-cluster): failed to insert some results into metric DB collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 error = Failed to check out connection to database: No backends found for this service file = oximeter/collector/src/results_sink.rs:92
93192025-04-30T06:44:09.488ZTRACtest_utilization (dns): buffer buffer = Length: 87 (0x57) bytes\n0000: ca c2 01 00 00 01 00 00 00 00 00 01 18 5f 63 6c ............._cl\n0010: 69 63 6b 68 6f 75 73 65 2d 61 64 6d 69 6e 2d 6b ickhouse-admin-k\n0020: 65 65 70 65 72 04 5f 74 63 70 0d 63 6f 6e 74 72 eeper._tcp.contr\n0030: 6f 6c 2d 70 6c 61 6e 65 05 6f 78 69 64 65 08 69 ol-plane.oxide.i\n0040: 6e 74 65 72 6e 61 6c 00 00 21 00 01 00 00 29 04 nternal..!....).\n0050: d0 00 00 00 00 00 00 ....... kind = dns peer_addr = [::1]:59821 req_id = de392a2d-0e1c-4463-b36d-8308160bce53
93202025-04-30T06:44:09.488ZDEBGtest_utilization (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 51906,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 24,\n 95,\n 99,\n 108,\n 105,\n 99,\n 107,\n 104,\n 111,\n 117,\n 115,\n 101,\n 45,\n 97,\n 100,\n 109,\n 105,\n 110,\n 45,\n 107,\n 101,\n 101,\n 112,\n 101,\n 114,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:59821 req_id = de392a2d-0e1c-4463-b36d-8308160bce53
93212025-04-30T06:44:09.489ZDEBGtest_utilization (store): query key key = _clickhouse-admin-keeper._tcp
93222025-04-30T06:44:09.489ZERROtest_utilization (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 51906, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 1, }, query: WireQuery { query: LowerQuery { name: LowerName( Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), ), original: Query { name: Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN, }, }, original: [ 24, 95, 99, 108, 105, 99, 107, 104, 111, 117, 115, 101, 45, 97, 100, 109, 105, 110, 45, 107, 101, 101, 112, 101, 114, 4, 95, 116, 99, 112, 13, 99, 111, 110, 116, 114, 111, 108, 45, 112, 108, 97, 110, 101, 5, 111, 120, 105, 100, 101, 8, 105, 110, 116, 101, 114, 110, 97, 108, 0, 0, 33, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: Some( Edns { rcode_high: 0, version: 0, dnssec_ok: false, max_payload: 1232, options: OPT { options: {}, }, }, ), } NXDOMAIN: no records found for name: "_clickhouse-admin-keeper._tcp.control-plane.oxide.internal." kind = dns peer_addr = [::1]:59821 req_id = de392a2d-0e1c-4463-b36d-8308160bce53
93232025-04-30T06:44:09.489ZDEBGtest_utilization (BackgroundTasks): begin collection background_task = inventory_collection
93242025-04-30T06:44:09.489ZDEBGtest_utilization (BackgroundTasks): begin collection from MGS background_task = inventory_collection mgs_url = http://[::1]:48759
93252025-04-30T06:44:09.489ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/ignition
93262025-04-30T06:44:09.494ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/ignition", status: 200, headers: {"content-type": "application/json", "x-request-id": "c5711d8c-1dca-4f91-859f-8de0c988da46", "content-length": "835", "date": "Wed, 30 Apr 2025 06:44:08 GMT"} })
93272025-04-30T06:44:09.494ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0
93282025-04-30T06:44:09.496ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "b002cd72-334b-467d-8082-d050fda5faa2", "content-length": "734", "date": "Wed, 30 Apr 2025 06:44:08 GMT"} })
93292025-04-30T06:44:09.496ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0/component/sp/caboose?firmware_slot=0
93302025-04-30T06:44:09.501ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "a27a2e15-3911-4bea-94e3-0cd44d75da4b", "content-length": "111", "date": "Wed, 30 Apr 2025 06:44:08 GMT"} })
93312025-04-30T06:44:09.501ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0/component/sp/caboose?firmware_slot=1
93322025-04-30T06:44:09.505ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "be1794d0-cc8d-4afd-b4b0-dd1822290750", "content-length": "111", "date": "Wed, 30 Apr 2025 06:44:08 GMT"} })
93332025-04-30T06:44:09.505ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0/component/rot/caboose?firmware_slot=0
93342025-04-30T06:44:09.510ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "c7dfb66f-de1e-4935-957c-8099861ab7b4", "content-length": "170", "date": "Wed, 30 Apr 2025 06:44:08 GMT"} })
93352025-04-30T06:44:09.510ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0/component/rot/caboose?firmware_slot=1
93362025-04-30T06:44:09.513ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "42700a6e-18e2-47ec-aa53-cad50bbaf0af", "content-length": "170", "date": "Wed, 30 Apr 2025 06:44:08 GMT"} })
93372025-04-30T06:44:09.513ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0/component/stage0/caboose?firmware_slot=0
93382025-04-30T06:44:09.516ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "9ac52765-1587-4194-ade7-1e8f9098d315", "content-length": "179", "date": "Wed, 30 Apr 2025 06:44:08 GMT"} })
93392025-04-30T06:44:09.517ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0/component/stage0/caboose?firmware_slot=1
93402025-04-30T06:44:09.521ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "feec17d2-e0ab-4ce0-8be6-bf7a14e3164e", "content-length": "179", "date": "Wed, 30 Apr 2025 06:44:08 GMT"} })
93412025-04-30T06:44:09.521ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0/component/rot/cmpa
93422025-04-30T06:44:09.522ZDEBGtest_utilization: attempt to fetch latest inventory collection result = Ok("found")
93432025-04-30T06:44:09.523ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "9af65088-f3f7-45f3-8b6d-88c74dcf1119", "content-length": "702", "date": "Wed, 30 Apr 2025 06:44:08 GMT"} })
93442025-04-30T06:44:09.526ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0/component/rot/cfpa
93452025-04-30T06:44:09.527ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "f744e52c-fbe3-4c8b-ac37-0789780533a1", "content-length": "727", "date": "Wed, 30 Apr 2025 06:44:08 GMT"} })
93462025-04-30T06:44:09.527ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0/component/rot/cfpa
93472025-04-30T06:44:09.528ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "461adfe7-0fa2-4422-be49-7c766e18e565", "content-length": "729", "date": "Wed, 30 Apr 2025 06:44:08 GMT"} })
93482025-04-30T06:44:09.528ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0/component/rot/cfpa
93492025-04-30T06:44:09.538ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "6b88bfef-68c6-4b2d-8f0c-8f658e83c89c", "content-length": "728", "date": "Wed, 30 Apr 2025 06:44:08 GMT"} })
93502025-04-30T06:44:09.538ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1
93512025-04-30T06:44:09.541ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1", status: 200, headers: {"content-type": "application/json", "x-request-id": "d4357491-b234-4857-a724-d86804348254", "content-length": "734", "date": "Wed, 30 Apr 2025 06:44:08 GMT"} })
93522025-04-30T06:44:09.541ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1/component/sp/caboose?firmware_slot=0
93532025-04-30T06:44:09.547ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "304e304b-8821-4071-afd4-f3fd181bb15a", "content-length": "111", "date": "Wed, 30 Apr 2025 06:44:08 GMT"} })
93542025-04-30T06:44:09.548ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1/component/sp/caboose?firmware_slot=1
93552025-04-30T06:44:09.551ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "31ecc3a9-c98a-456e-a614-ed318079c652", "content-length": "111", "date": "Wed, 30 Apr 2025 06:44:08 GMT"} })
93562025-04-30T06:44:09.552ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1/component/rot/caboose?firmware_slot=0
93572025-04-30T06:44:09.557ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "a0b91ea0-9bfb-433c-bacc-cad6f80690bf", "content-length": "170", "date": "Wed, 30 Apr 2025 06:44:08 GMT"} })
93582025-04-30T06:44:09.557ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1/component/rot/caboose?firmware_slot=1
93592025-04-30T06:44:09.563ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "79d3378f-b15f-4b4b-b1ed-6f7d692e565d", "content-length": "170", "date": "Wed, 30 Apr 2025 06:44:08 GMT"} })
93602025-04-30T06:44:09.563ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1/component/stage0/caboose?firmware_slot=0
93612025-04-30T06:44:09.573ZTRACtest_utilization (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory
93622025-04-30T06:44:09.574ZTRACtest_utilization (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
93632025-04-30T06:44:09.574ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
93642025-04-30T06:44:09.574ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
93652025-04-30T06:44:09.575ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
93662025-04-30T06:44:09.584ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
93672025-04-30T06:44:09.593ZDEBGtest_utilization (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory result = Ok(())
93682025-04-30T06:44:09.593ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
93692025-04-30T06:44:09.593ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
93702025-04-30T06:44:09.593ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
93712025-04-30T06:44:09.604ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
93722025-04-30T06:44:09.604ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
93732025-04-30T06:44:09.606ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
93742025-04-30T06:44:09.667ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "b712cd95-29ca-4749-8dcb-2adeda8c59e5", "content-length": "179", "date": "Wed, 30 Apr 2025 06:44:08 GMT"} })
93752025-04-30T06:44:09.669ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1/component/stage0/caboose?firmware_slot=1
93762025-04-30T06:44:09.677ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "24d26fbd-874e-4207-bc90-1d3e51cf5cf1", "content-length": "179", "date": "Wed, 30 Apr 2025 06:44:09 GMT"} })
93772025-04-30T06:44:09.678ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1/component/rot/cmpa
93782025-04-30T06:44:09.679ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "ff724b83-08dd-4f8d-acf2-0652b081023e", "content-length": "702", "date": "Wed, 30 Apr 2025 06:44:09 GMT"} })
93792025-04-30T06:44:09.679ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1/component/rot/cfpa
93802025-04-30T06:44:09.684ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "02de5929-743e-4ab4-b90c-064df5c47273", "content-length": "727", "date": "Wed, 30 Apr 2025 06:44:09 GMT"} })
93812025-04-30T06:44:09.685ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1/component/rot/cfpa
93822025-04-30T06:44:09.687ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "56ce204a-60ef-462d-a5ff-fa249488120f", "content-length": "729", "date": "Wed, 30 Apr 2025 06:44:09 GMT"} })
93832025-04-30T06:44:09.687ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1/component/rot/cfpa
93842025-04-30T06:44:09.688ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "fc44829d-2fff-4cd5-9ca1-d39ad6fb3509", "content-length": "728", "date": "Wed, 30 Apr 2025 06:44:09 GMT"} })
93852025-04-30T06:44:09.688ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0
93862025-04-30T06:44:09.692ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "32bc0f08-159f-4920-b744-2f9896e269ee", "content-length": "723", "date": "Wed, 30 Apr 2025 06:44:09 GMT"} })
93872025-04-30T06:44:09.692ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0/component/sp/caboose?firmware_slot=0
93882025-04-30T06:44:09.694ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "06d79645-2991-476f-a8bb-3deedf5085f6", "content-length": "109", "date": "Wed, 30 Apr 2025 06:44:09 GMT"} })
93892025-04-30T06:44:09.694ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0/component/sp/caboose?firmware_slot=1
93902025-04-30T06:44:09.696ZDEBGtest_utilization: attempt to fetch latest inventory collection result = Ok("found")
93912025-04-30T06:44:09.697ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "5ef57d98-9b18-4b4d-8a5c-e96cc57f802b", "content-length": "109", "date": "Wed, 30 Apr 2025 06:44:09 GMT"} })
93922025-04-30T06:44:09.698ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0/component/rot/caboose?firmware_slot=0
93932025-04-30T06:44:09.699ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "a3b57c56-afd6-40d3-a67b-fd1e467277b4", "content-length": "169", "date": "Wed, 30 Apr 2025 06:44:09 GMT"} })
93942025-04-30T06:44:09.699ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0/component/rot/caboose?firmware_slot=1
93952025-04-30T06:44:09.703ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "5523af4a-ae43-4843-a31b-b13b37a673bb", "content-length": "169", "date": "Wed, 30 Apr 2025 06:44:09 GMT"} })
93962025-04-30T06:44:09.703ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0/component/stage0/caboose?firmware_slot=0
93972025-04-30T06:44:09.705ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "e3f5c3bb-0236-4ee5-9852-82b83b08850f", "content-length": "178", "date": "Wed, 30 Apr 2025 06:44:09 GMT"} })
93982025-04-30T06:44:09.705ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0/component/stage0/caboose?firmware_slot=1
93992025-04-30T06:44:09.707ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "3ae3a34b-f70d-4cd6-9baf-7a345da3f18b", "content-length": "178", "date": "Wed, 30 Apr 2025 06:44:09 GMT"} })
94002025-04-30T06:44:09.708ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0/component/rot/cmpa
94012025-04-30T06:44:09.708ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "f77aa015-602d-4841-9f3c-7763fd629edf", "content-length": "702", "date": "Wed, 30 Apr 2025 06:44:09 GMT"} })
94022025-04-30T06:44:09.709ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0/component/rot/cfpa
94032025-04-30T06:44:09.710ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "60ce88a4-622d-4aff-b883-4e7239d66390", "content-length": "727", "date": "Wed, 30 Apr 2025 06:44:09 GMT"} })
94042025-04-30T06:44:09.710ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0/component/rot/cfpa
94052025-04-30T06:44:09.711ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "9bc76358-28cb-4ea4-82c4-6ae70cd5f365", "content-length": "729", "date": "Wed, 30 Apr 2025 06:44:09 GMT"} })
94062025-04-30T06:44:09.711ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0/component/rot/cfpa
94072025-04-30T06:44:09.713ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "01973567-f23f-403a-9b01-e395e734d7f5", "content-length": "728", "date": "Wed, 30 Apr 2025 06:44:09 GMT"} })
94082025-04-30T06:44:09.714ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1
94092025-04-30T06:44:09.715ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1", status: 200, headers: {"content-type": "application/json", "x-request-id": "4f2be971-f617-4566-865d-494ff03cc251", "content-length": "723", "date": "Wed, 30 Apr 2025 06:44:09 GMT"} })
94102025-04-30T06:44:09.715ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1/component/sp/caboose?firmware_slot=0
94112025-04-30T06:44:09.717ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "3207cddc-8f78-4c2e-bbc4-1daec5abb31a", "content-length": "109", "date": "Wed, 30 Apr 2025 06:44:09 GMT"} })
94122025-04-30T06:44:09.717ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1/component/sp/caboose?firmware_slot=1
94132025-04-30T06:44:09.719ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "6add6a87-e95a-4a2b-873d-5fb71c139895", "content-length": "109", "date": "Wed, 30 Apr 2025 06:44:09 GMT"} })
94142025-04-30T06:44:09.719ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1/component/rot/caboose?firmware_slot=0
94152025-04-30T06:44:09.722ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "7e3081d3-59a2-4b1d-bd03-db65c3da79c7", "content-length": "169", "date": "Wed, 30 Apr 2025 06:44:09 GMT"} })
94162025-04-30T06:44:09.722ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1/component/rot/caboose?firmware_slot=1
94172025-04-30T06:44:09.726ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "e95ee4b1-3587-47e8-a449-4010a5e559f2", "content-length": "169", "date": "Wed, 30 Apr 2025 06:44:09 GMT"} })
94182025-04-30T06:44:09.726ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1/component/stage0/caboose?firmware_slot=0
94192025-04-30T06:44:09.729ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "46396ba5-df22-4ba0-8bfe-40ee9b85e854", "content-length": "178", "date": "Wed, 30 Apr 2025 06:44:09 GMT"} })
94202025-04-30T06:44:09.729ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1/component/stage0/caboose?firmware_slot=1
94212025-04-30T06:44:09.732ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "a83fe3d7-9699-41e0-9391-11bdbac8e916", "content-length": "178", "date": "Wed, 30 Apr 2025 06:44:09 GMT"} })
94222025-04-30T06:44:09.732ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1/component/rot/cmpa
94232025-04-30T06:44:09.733ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "c2d57dd6-2688-40bd-8046-545d99e4ecb7", "content-length": "702", "date": "Wed, 30 Apr 2025 06:44:09 GMT"} })
94242025-04-30T06:44:09.733ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1/component/rot/cfpa
94252025-04-30T06:44:09.735ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "e099719e-14db-41f9-8b1f-5e217cbef3a9", "content-length": "727", "date": "Wed, 30 Apr 2025 06:44:09 GMT"} })
94262025-04-30T06:44:09.736ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1/component/rot/cfpa
94272025-04-30T06:44:09.737ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "ce6621c6-247d-433c-b7da-cdd565ff4fd4", "content-length": "729", "date": "Wed, 30 Apr 2025 06:44:09 GMT"} })
94282025-04-30T06:44:09.737ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1/component/rot/cfpa
94292025-04-30T06:44:09.739ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "74661c38-a84e-4666-98fa-9cdfdd8e1654", "content-length": "728", "date": "Wed, 30 Apr 2025 06:44:09 GMT"} })
94302025-04-30T06:44:09.739ZTRACtest_utilization (BackgroundTasks): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet
94312025-04-30T06:44:09.740ZTRACtest_utilization (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
94322025-04-30T06:44:09.740ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
94332025-04-30T06:44:09.740ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
94342025-04-30T06:44:09.741ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
94352025-04-30T06:44:09.744ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
94362025-04-30T06:44:09.749ZDEBGtest_utilization (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet result = Ok(())
94372025-04-30T06:44:09.749ZTRACtest_utilization (BackgroundTasks): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet
94382025-04-30T06:44:09.749ZTRACtest_utilization (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
94392025-04-30T06:44:09.749ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
94402025-04-30T06:44:09.749ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
94412025-04-30T06:44:09.750ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
94422025-04-30T06:44:09.750ZTRACtest_utilization (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory
94432025-04-30T06:44:09.751ZTRACtest_utilization (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
94442025-04-30T06:44:09.751ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
94452025-04-30T06:44:09.751ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
94462025-04-30T06:44:09.751ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
94472025-04-30T06:44:09.754ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
94482025-04-30T06:44:09.757ZDEBGtest_utilization (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet result = Ok(())
94492025-04-30T06:44:09.758ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
94502025-04-30T06:44:09.758ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
94512025-04-30T06:44:09.758ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
94522025-04-30T06:44:09.760ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
94532025-04-30T06:44:09.766ZDEBGtest_utilization (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory result = Ok(())
94542025-04-30T06:44:09.766ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
94552025-04-30T06:44:09.766ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
94562025-04-30T06:44:09.767ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
94572025-04-30T06:44:09.840ZDEBGtest_utilization (BackgroundTasks): begin collection from Sled Agent background_task = inventory_collection sled_agent_url = http://[::1]:39365
94582025-04-30T06:44:09.840ZDEBGtest_utilization (BackgroundTasks): client request SledAgent = http://[::1]:39365 background_task = inventory_collection body = None method = GET uri = http://[::1]:39365/inventory
94592025-04-30T06:44:09.842ZINFOtest_utilization (dropshot): accepted connection local_addr = [::1]:39365 remote_addr = [::1]:38378 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
94602025-04-30T06:44:09.842ZTRACtest_utilization (dropshot): incoming request local_addr = [::1]:39365 method = GET remote_addr = [::1]:38378 req_id = 9288364f-74f9-4e1e-a3cb-7600ecde3113 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /inventory
94612025-04-30T06:44:09.844ZINFOtest_utilization (dropshot): request completed latency_us = 2210 local_addr = [::1]:39365 method = GET remote_addr = [::1]:38378 req_id = 9288364f-74f9-4e1e-a3cb-7600ecde3113 response_code = 200 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /inventory
94622025-04-30T06:44:09.844ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
94632025-04-30T06:44:09.844ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
94642025-04-30T06:44:09.845ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
94652025-04-30T06:44:09.846ZDEBGtest_utilization (BackgroundTasks): client response SledAgent = http://[::1]:39365 background_task = inventory_collection result = Ok(Response { url: "http://[::1]:39365/inventory", status: 200, headers: {"content-type": "application/json", "x-request-id": "9288364f-74f9-4e1e-a3cb-7600ecde3113", "content-length": "3141", "date": "Wed, 30 Apr 2025 06:44:09 GMT"} })
94662025-04-30T06:44:09.851ZDEBGtest_utilization (BackgroundTasks): begin collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
94672025-04-30T06:44:09.851ZDEBGtest_utilization (BackgroundTasks): end collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
94682025-04-30T06:44:09.851ZDEBGtest_utilization (BackgroundTasks): finished collection background_task = inventory_collection
94692025-04-30T06:44:09.854ZTRACtest_utilization (BackgroundTasks): authorize begin action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory
94702025-04-30T06:44:09.854ZTRACtest_utilization (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
94712025-04-30T06:44:09.854ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
94722025-04-30T06:44:09.854ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
94732025-04-30T06:44:09.855ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
94742025-04-30T06:44:09.873ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
94752025-04-30T06:44:09.879ZDEBGtest_utilization (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory result = Ok(())
94762025-04-30T06:44:09.879ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
94772025-04-30T06:44:09.879ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
94782025-04-30T06:44:09.880ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
94792025-04-30T06:44:09.920ZDEBGtest_utilization: attempt to fetch latest inventory collection result = Ok("found")
94802025-04-30T06:44:09.923ZWARNtest_utilization (datastore): Non-retryable transaction failure err = DatabaseError(UniqueViolation, "duplicate key value violates unique constraint \\"inv_caboose_pkey\\"") transaction = inventory_insert_collection
94812025-04-30T06:44:09.924ZWARNtest_utilization (BackgroundTasks): inventory collection failed background_task = inventory_collection error = failed to collect inventory: saving inventory to database: Internal Error: unexpected database error: duplicate key value violates unique constraint "inv_caboose_pkey"
94822025-04-30T06:44:09.924ZDEBGtest_utilization (BackgroundTasks): activation complete background_task = inventory_collection elapsed = 618.389822ms iteration = 19 status = {"error":"failed to collect inventory: saving inventory to database: Internal Error: unexpected database error: duplicate key value violates unique constraint \\"inv_caboose_pkey\\""}
94832025-04-30T06:44:09.924ZDEBGtest_utilization (BackgroundTasks): activating background_task = inventory_collection iteration = 20 reason = Signaled
94842025-04-30T06:44:09.924ZTRACtest_utilization (BackgroundTasks): authorize begin action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory
94852025-04-30T06:44:09.924ZTRACtest_utilization (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
94862025-04-30T06:44:09.924ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
94872025-04-30T06:44:09.924ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
94882025-04-30T06:44:09.925ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
94892025-04-30T06:44:09.929ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
94902025-04-30T06:44:09.934ZDEBGtest_utilization (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory result = Ok(())
94912025-04-30T06:44:09.934ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
94922025-04-30T06:44:09.934ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
94932025-04-30T06:44:09.934ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
94942025-04-30T06:44:09.938ZDEBGtest_utilization (BackgroundTasks): inventory_prune_one: nothing eligible for removal (too few) background_task = inventory_collection candidates = [(13d8b8d9-6c6b-47a6-9190-952544edb75b, 0)]
94952025-04-30T06:44:09.938ZTRACtest_utilization (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal
94962025-04-30T06:44:09.938ZTRACtest_utilization (dns): buffer buffer = Length: 67 (0x43) bytes\n0000: c8 3e 01 00 00 01 00 00 00 00 00 01 04 5f 6d 67 .>..........._mg\n0010: 73 04 5f 74 63 70 0d 63 6f 6e 74 72 6f 6c 2d 70 s._tcp.control-p\n0020: 6c 61 6e 65 05 6f 78 69 64 65 08 69 6e 74 65 72 lane.oxide.inter\n0030: 6e 61 6c 00 00 21 00 01 00 00 29 04 d0 00 00 00 nal..!....).....\n0040: 00 00 00 ... kind = dns peer_addr = [::1]:62186 req_id = 1057ea50-1fea-49e0-9137-45cbbc74578d
94972025-04-30T06:44:09.939ZDEBGtest_utilization (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 51262,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_mgs._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_mgs._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 4,\n 95,\n 109,\n 103,\n 115,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:62186 req_id = 1057ea50-1fea-49e0-9137-45cbbc74578d
94982025-04-30T06:44:09.939ZDEBGtest_utilization (store): query key key = _mgs._tcp
94992025-04-30T06:44:09.939ZDEBGtest_utilization (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
95002025-04-30T06:44:09.940ZDEBGtest_utilization (dns): dns response additional_records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal"), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] kind = dns peer_addr = [::1]:62186 query = LowerQuery { name: LowerName(Name("_mgs._tcp.control-plane.oxide.internal.")), original: Query { name: Name("_mgs._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN } } records = [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 48759, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal") })) }] req_id = 1057ea50-1fea-49e0-9137-45cbbc74578d
95012025-04-30T06:44:09.941ZDEBGtest_utilization (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgs._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 48759, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }], valid_until: Instant { tv_sec: 4314, tv_nsec: 739735163 } })
95022025-04-30T06:44:09.941ZTRACtest_utilization (DnsResolver): lookup_service_targets: looking up SRV target
95032025-04-30T06:44:09.942ZTRACtest_utilization (dns): buffer buffer = Length: 108 (0x6c) bytes\n0000: 4a 50 01 00 00 01 00 00 00 00 00 01 2d 64 65 6e JP..........-den\n0010: 64 72 69 74 65 2d 62 36 64 36 35 33 34 31 2d 31 drite-b6d65341-1\n0020: 36 37 63 2d 34 31 64 66 2d 39 62 35 63 2d 34 31 67c-41df-9b5c-41\n0030: 63 64 65 64 39 39 63 32 32 39 04 68 6f 73 74 0d cded99c229.host.\n0040: 63 6f 6e 74 72 6f 6c 2d 70 6c 61 6e 65 05 6f 78 control-plane.ox\n0050: 69 64 65 08 69 6e 74 65 72 6e 61 6c 00 00 1c 00 ide.internal....\n0060: 01 00 00 29 04 d0 00 00 00 00 00 00 ...)........ kind = dns peer_addr = [::1]:63970 req_id = 260c9e26-a216-4bd4-ae56-e9e256e1b13b
95042025-04-30T06:44:09.943ZDEBGtest_utilization (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 19024,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n query_type: AAAA,\n query_class: IN,\n },\n },\n original: [\n 45,\n 100,\n 101,\n 110,\n 100,\n 114,\n 105,\n 116,\n 101,\n 45,\n 98,\n 54,\n 100,\n 54,\n 53,\n 51,\n 52,\n 49,\n 45,\n 49,\n 54,\n 55,\n 99,\n 45,\n 52,\n 49,\n 100,\n 102,\n 45,\n 57,\n 98,\n 53,\n 99,\n 45,\n 52,\n 49,\n 99,\n 100,\n 101,\n 100,\n 57,\n 57,\n 99,\n 50,\n 50,\n 57,\n 4,\n 104,\n 111,\n 115,\n 116,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 28,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:63970 req_id = 260c9e26-a216-4bd4-ae56-e9e256e1b13b
95052025-04-30T06:44:09.944ZDEBGtest_utilization (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
95062025-04-30T06:44:09.944ZDEBGtest_utilization (dns): dns response additional_records = [] kind = dns peer_addr = [::1]:63970 query = LowerQuery { name: LowerName(Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.")), original: Query { name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), query_type: AAAA, query_class: IN } } records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] req_id = 260c9e26-a216-4bd4-ae56-e9e256e1b13b
95072025-04-30T06:44:10.007ZTRACtest_utilization (DnsResolver): lookup_all_socket_v6 srv dns_name = _clickhouse-admin-keeper._tcp.control-plane.oxide.internal
95082025-04-30T06:44:10.008ZTRACtest_utilization (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory
95092025-04-30T06:44:10.008ZTRACtest_utilization (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
95102025-04-30T06:44:10.008ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
95112025-04-30T06:44:10.008ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
95122025-04-30T06:44:10.009ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
95132025-04-30T06:44:10.009ZINFOtest_utilization (oximeter-agent): refreshing list of producers from Nexus collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 file = oximeter/collector/src/agent.rs:562
95142025-04-30T06:44:10.009ZDEBGtest_utilization (collection-task): sent timer-based collection request to the collection task collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
95152025-04-30T06:44:10.010ZDEBGtest_utilization (collection-task): collecting from producer collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
95162025-04-30T06:44:10.011ZDEBGtest_utilization (oximeter): client request body = None method = GET uri = http://127.0.0.1:32956/metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?sort_by=id_ascending
95172025-04-30T06:44:10.013ZTRACtest_utilization (dropshot_internal): incoming request local_addr = 127.0.0.1:32956 method = GET remote_addr = 127.0.0.1:55797 req_id = 99d8d7f8-6eff-40d6-b9c5-aee3984571bf uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?sort_by=id_ascending
95182025-04-30T06:44:10.013ZTRACtest_utilization (dns): buffer buffer = Length: 87 (0x57) bytes\n0000: 80 88 01 00 00 01 00 00 00 00 00 01 18 5f 63 6c ............._cl\n0010: 69 63 6b 68 6f 75 73 65 2d 61 64 6d 69 6e 2d 6b ickhouse-admin-k\n0020: 65 65 70 65 72 04 5f 74 63 70 0d 63 6f 6e 74 72 eeper._tcp.contr\n0030: 6f 6c 2d 70 6c 61 6e 65 05 6f 78 69 64 65 08 69 ol-plane.oxide.i\n0040: 6e 74 65 72 6e 61 6c 00 00 21 00 01 00 00 29 04 nternal..!....).\n0050: d0 00 00 00 00 00 00 ....... kind = dns peer_addr = [::1]:58110 req_id = c9524155-256f-4682-b486-ebe058382d3b
95192025-04-30T06:44:10.013ZDEBGtest_utilization (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 32904,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 24,\n 95,\n 99,\n 108,\n 105,\n 99,\n 107,\n 104,\n 111,\n 117,\n 115,\n 101,\n 45,\n 97,\n 100,\n 109,\n 105,\n 110,\n 45,\n 107,\n 101,\n 101,\n 112,\n 101,\n 114,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:58110 req_id = c9524155-256f-4682-b486-ebe058382d3b
95202025-04-30T06:44:10.013ZDEBGtest_utilization (store): query key key = _clickhouse-admin-keeper._tcp
95212025-04-30T06:44:10.014ZERROtest_utilization (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 32904, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 1, }, query: WireQuery { query: LowerQuery { name: LowerName( Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), ), original: Query { name: Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN, }, }, original: [ 24, 95, 99, 108, 105, 99, 107, 104, 111, 117, 115, 101, 45, 97, 100, 109, 105, 110, 45, 107, 101, 101, 112, 101, 114, 4, 95, 116, 99, 112, 13, 99, 111, 110, 116, 114, 111, 108, 45, 112, 108, 97, 110, 101, 5, 111, 120, 105, 100, 101, 8, 105, 110, 116, 101, 114, 110, 97, 108, 0, 0, 33, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: Some( Edns { rcode_high: 0, version: 0, dnssec_ok: false, max_payload: 1232, options: OPT { options: {}, }, }, ), } NXDOMAIN: no records found for name: "_clickhouse-admin-keeper._tcp.control-plane.oxide.internal." kind = dns peer_addr = [::1]:58110 req_id = c9524155-256f-4682-b486-ebe058382d3b
95222025-04-30T06:44:10.014ZTRACtest_utilization (dropshot_internal): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = 127.0.0.1:32956 method = GET remote_addr = 127.0.0.1:55797 req_id = 99d8d7f8-6eff-40d6-b9c5-aee3984571bf resource = Database uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?sort_by=id_ascending
95232025-04-30T06:44:10.014ZDEBGtest_utilization (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = 127.0.0.1:32956 method = GET remote_addr = 127.0.0.1:55797 req_id = 99d8d7f8-6eff-40d6-b9c5-aee3984571bf roles = RoleSet { roles: {} } uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?sort_by=id_ascending
95242025-04-30T06:44:10.015ZDEBGtest_utilization (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = 127.0.0.1:32956 method = GET remote_addr = 127.0.0.1:55797 req_id = 99d8d7f8-6eff-40d6-b9c5-aee3984571bf resource = Database result = Ok(()) uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?sort_by=id_ascending
95252025-04-30T06:44:10.017ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
95262025-04-30T06:44:10.024ZDEBGtest_utilization (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory result = Ok(())
95272025-04-30T06:44:10.024ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
95282025-04-30T06:44:10.024ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
95292025-04-30T06:44:10.024ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
95302025-04-30T06:44:10.026ZDEBGtest_utilization (BackgroundTasks): begin collection background_task = inventory_collection
95312025-04-30T06:44:10.026ZDEBGtest_utilization (BackgroundTasks): begin collection from MGS background_task = inventory_collection mgs_url = http://[::1]:48759
95322025-04-30T06:44:10.026ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/ignition
95332025-04-30T06:44:10.027ZTRACtest_utilization (collection-task): sent collection request to producer collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
95342025-04-30T06:44:10.029ZDEBGtest_utilization (collection-task): collected results from producer collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 n_results = 1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
95352025-04-30T06:44:10.029ZTRACtest_utilization (collection-task): forwarded results to main collection loop collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
95362025-04-30T06:44:10.029ZTRACtest_utilization (collection-task): top of inner collection loop, waiting for next request collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
95372025-04-30T06:44:10.032ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
95382025-04-30T06:44:10.032ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
95392025-04-30T06:44:10.033ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
95402025-04-30T06:44:10.035ZINFOtest_utilization (dropshot_internal): request completed latency_us = 22416 local_addr = 127.0.0.1:32956 method = GET remote_addr = 127.0.0.1:55797 req_id = 99d8d7f8-6eff-40d6-b9c5-aee3984571bf response_code = 200 uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?sort_by=id_ascending
95412025-04-30T06:44:10.036ZDEBGtest_utilization (oximeter): client response result = Ok(Response { url: "http://127.0.0.1:32956/metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?sort_by=id_ascending", status: 200, headers: {"content-type": "application/json", "x-request-id": "99d8d7f8-6eff-40d6-b9c5-aee3984571bf", "content-length": "405", "date": "Wed, 30 Apr 2025 06:44:09 GMT"} })
95422025-04-30T06:44:10.036ZDEBGtest_utilization (oximeter): client request body = None method = GET uri = http://127.0.0.1:32956/metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJhYzU1ZDhkOS1hMjk4LTQ1MzQtODk4Yi1iZWVlMWM5ZTRlNTIifX0%3D
95432025-04-30T06:44:10.038ZTRACtest_utilization (dropshot_internal): incoming request local_addr = 127.0.0.1:32956 method = GET remote_addr = 127.0.0.1:55797 req_id = fb69e10d-bdc4-4fe3-a324-5dcc0297652c uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJhYzU1ZDhkOS1hMjk4LTQ1MzQtODk4Yi1iZWVlMWM5ZTRlNTIifX0%3D
95442025-04-30T06:44:10.039ZTRACtest_utilization (dropshot_internal): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = 127.0.0.1:32956 method = GET remote_addr = 127.0.0.1:55797 req_id = fb69e10d-bdc4-4fe3-a324-5dcc0297652c resource = Database uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJhYzU1ZDhkOS1hMjk4LTQ1MzQtODk4Yi1iZWVlMWM5ZTRlNTIifX0%3D
95452025-04-30T06:44:10.039ZDEBGtest_utilization (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = 127.0.0.1:32956 method = GET remote_addr = 127.0.0.1:55797 req_id = fb69e10d-bdc4-4fe3-a324-5dcc0297652c roles = RoleSet { roles: {} } uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJhYzU1ZDhkOS1hMjk4LTQ1MzQtODk4Yi1iZWVlMWM5ZTRlNTIifX0%3D
95462025-04-30T06:44:10.039ZDEBGtest_utilization (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = 127.0.0.1:32956 method = GET remote_addr = 127.0.0.1:55797 req_id = fb69e10d-bdc4-4fe3-a324-5dcc0297652c resource = Database result = Ok(()) uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJhYzU1ZDhkOS1hMjk4LTQ1MzQtODk4Yi1iZWVlMWM5ZTRlNTIifX0%3D
95472025-04-30T06:44:10.043ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/ignition", status: 200, headers: {"content-type": "application/json", "x-request-id": "6d94470c-65c3-44af-a769-a3840d06c462", "content-length": "835", "date": "Wed, 30 Apr 2025 06:44:09 GMT"} })
95482025-04-30T06:44:10.044ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0
95492025-04-30T06:44:10.045ZINFOtest_utilization (dropshot_internal): request completed latency_us = 6308 local_addr = 127.0.0.1:32956 method = GET remote_addr = 127.0.0.1:55797 req_id = fb69e10d-bdc4-4fe3-a324-5dcc0297652c response_code = 200 uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJhYzU1ZDhkOS1hMjk4LTQ1MzQtODk4Yi1iZWVlMWM5ZTRlNTIifX0%3D
95502025-04-30T06:44:10.045ZDEBGtest_utilization (oximeter): client response result = Ok(Response { url: "http://127.0.0.1:32956/metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJhYzU1ZDhkOS1hMjk4LTQ1MzQtODk4Yi1iZWVlMWM5ZTRlNTIifX0%3D", status: 200, headers: {"content-type": "application/json", "x-request-id": "fb69e10d-bdc4-4fe3-a324-5dcc0297652c", "content-length": "29", "date": "Wed, 30 Apr 2025 06:44:09 GMT"} })
95512025-04-30T06:44:10.045ZTRACtest_utilization (oximeter-agent): ignoring request to update existing metric producer, since the endpoint information is the same as the existing collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
95522025-04-30T06:44:10.045ZTRACtest_utilization (oximeter-agent): ignoring request to update existing metric producer, since the endpoint information is the same as the existing collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = ac55d8d9-a298-4534-898b-beee1c9e4e52
95532025-04-30T06:44:10.045ZINFOtest_utilization (oximeter-agent): refreshed list of producers from Nexus collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 file = oximeter/collector/src/agent.rs:637 n_current_tasks = 2 n_pruned_tasks = 0
95542025-04-30T06:44:10.048ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "9d67c4bd-cb80-41dd-a5e8-0b4e52bc6bc2", "content-length": "734", "date": "Wed, 30 Apr 2025 06:44:09 GMT"} })
95552025-04-30T06:44:10.048ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0/component/sp/caboose?firmware_slot=0
95562025-04-30T06:44:10.053ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "484862d2-4a93-4453-94eb-60ad7dfa81d3", "content-length": "111", "date": "Wed, 30 Apr 2025 06:44:09 GMT"} })
95572025-04-30T06:44:10.053ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0/component/sp/caboose?firmware_slot=1
95582025-04-30T06:44:10.059ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "6183d041-8075-40c1-b5f0-4906536d430d", "content-length": "111", "date": "Wed, 30 Apr 2025 06:44:09 GMT"} })
95592025-04-30T06:44:10.060ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0/component/rot/caboose?firmware_slot=0
95602025-04-30T06:44:10.072ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "461f70b8-7be7-4b43-b89a-d4920819e8b1", "content-length": "170", "date": "Wed, 30 Apr 2025 06:44:09 GMT"} })
95612025-04-30T06:44:10.073ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0/component/rot/caboose?firmware_slot=1
95622025-04-30T06:44:10.076ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "3a1731f2-2015-4c93-9120-5fe899acf375", "content-length": "170", "date": "Wed, 30 Apr 2025 06:44:09 GMT"} })
95632025-04-30T06:44:10.076ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0/component/stage0/caboose?firmware_slot=0
95642025-04-30T06:44:10.079ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "239cb472-2c05-44bd-a3d6-db7702f8f739", "content-length": "179", "date": "Wed, 30 Apr 2025 06:44:09 GMT"} })
95652025-04-30T06:44:10.079ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0/component/stage0/caboose?firmware_slot=1
95662025-04-30T06:44:10.084ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "cb633801-44c4-474c-aec6-bed4608ade48", "content-length": "179", "date": "Wed, 30 Apr 2025 06:44:09 GMT"} })
95672025-04-30T06:44:10.084ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0/component/rot/cmpa
95682025-04-30T06:44:10.133ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "58bb1ce9-a7a1-463b-8aa8-761bf9ea3aa1", "content-length": "702", "date": "Wed, 30 Apr 2025 06:44:09 GMT"} })
95692025-04-30T06:44:10.134ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0/component/rot/cfpa
95702025-04-30T06:44:10.134ZDEBGtest_utilization: attempt to fetch latest inventory collection result = Ok("found")
95712025-04-30T06:44:10.146ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "f8e21d90-9918-4c6f-bd8b-6d4b449fcd3d", "content-length": "727", "date": "Wed, 30 Apr 2025 06:44:09 GMT"} })
95722025-04-30T06:44:10.147ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0/component/rot/cfpa
95732025-04-30T06:44:10.148ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "e22eb18e-24cc-4ee7-aea8-f07107618886", "content-length": "729", "date": "Wed, 30 Apr 2025 06:44:09 GMT"} })
95742025-04-30T06:44:10.148ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0/component/rot/cfpa
95752025-04-30T06:44:10.149ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "75211796-dc1d-4a6f-bbca-85e07a7c04bb", "content-length": "728", "date": "Wed, 30 Apr 2025 06:44:09 GMT"} })
95762025-04-30T06:44:10.149ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1
95772025-04-30T06:44:10.152ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1", status: 200, headers: {"content-type": "application/json", "x-request-id": "82cfef9b-839f-43be-acb1-e78d520eb1bb", "content-length": "734", "date": "Wed, 30 Apr 2025 06:44:09 GMT"} })
95782025-04-30T06:44:10.156ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1/component/sp/caboose?firmware_slot=0
95792025-04-30T06:44:10.166ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "8e2226e7-4f06-48d5-8949-cd77a7a9c216", "content-length": "111", "date": "Wed, 30 Apr 2025 06:44:09 GMT"} })
95802025-04-30T06:44:10.166ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1/component/sp/caboose?firmware_slot=1
95812025-04-30T06:44:10.170ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "bedb8168-8b9c-4d51-a6ec-d85f00c4c7b1", "content-length": "111", "date": "Wed, 30 Apr 2025 06:44:09 GMT"} })
95822025-04-30T06:44:10.172ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1/component/rot/caboose?firmware_slot=0
95832025-04-30T06:44:10.176ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "cc47a31b-aaf0-4efc-9bb1-f793d20a827f", "content-length": "170", "date": "Wed, 30 Apr 2025 06:44:09 GMT"} })
95842025-04-30T06:44:10.176ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1/component/rot/caboose?firmware_slot=1
95852025-04-30T06:44:10.180ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "3ae57fb9-d591-4caa-877b-0aff997dca59", "content-length": "170", "date": "Wed, 30 Apr 2025 06:44:09 GMT"} })
95862025-04-30T06:44:10.180ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1/component/stage0/caboose?firmware_slot=0
95872025-04-30T06:44:10.182ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "62634356-248d-4604-b39c-f3e7d47454c3", "content-length": "179", "date": "Wed, 30 Apr 2025 06:44:09 GMT"} })
95882025-04-30T06:44:10.186ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1/component/stage0/caboose?firmware_slot=1
95892025-04-30T06:44:10.186ZTRACtest_utilization (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory
95902025-04-30T06:44:10.186ZTRACtest_utilization (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
95912025-04-30T06:44:10.186ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
95922025-04-30T06:44:10.186ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
95932025-04-30T06:44:10.187ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
95942025-04-30T06:44:10.192ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
95952025-04-30T06:44:10.202ZDEBGtest_utilization (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory result = Ok(())
95962025-04-30T06:44:10.202ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
95972025-04-30T06:44:10.202ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
95982025-04-30T06:44:10.204ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
95992025-04-30T06:44:10.204ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "76d016b6-799d-4ca6-be96-fdf1a2d213e0", "content-length": "179", "date": "Wed, 30 Apr 2025 06:44:09 GMT"} })
96002025-04-30T06:44:10.204ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1/component/rot/cmpa
96012025-04-30T06:44:10.211ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
96022025-04-30T06:44:10.211ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
96032025-04-30T06:44:10.211ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
96042025-04-30T06:44:10.211ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "bb6ad052-fad5-47c6-8dee-42d5c08f4483", "content-length": "702", "date": "Wed, 30 Apr 2025 06:44:09 GMT"} })
96052025-04-30T06:44:10.212ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1/component/rot/cfpa
96062025-04-30T06:44:10.213ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "f85e4cf8-dc66-4e64-8483-53bf955a24cf", "content-length": "727", "date": "Wed, 30 Apr 2025 06:44:09 GMT"} })
96072025-04-30T06:44:10.213ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1/component/rot/cfpa
96082025-04-30T06:44:10.214ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "c87d9d12-cc72-4fed-b4b6-c50642b3b6c7", "content-length": "729", "date": "Wed, 30 Apr 2025 06:44:09 GMT"} })
96092025-04-30T06:44:10.215ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1/component/rot/cfpa
96102025-04-30T06:44:10.217ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "ad3882b8-0f03-432e-9b73-5352fc3296f8", "content-length": "728", "date": "Wed, 30 Apr 2025 06:44:09 GMT"} })
96112025-04-30T06:44:10.217ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0
96122025-04-30T06:44:10.220ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "5de48a7d-a6ef-4236-9668-988b8a1c661e", "content-length": "723", "date": "Wed, 30 Apr 2025 06:44:09 GMT"} })
96132025-04-30T06:44:10.220ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0/component/sp/caboose?firmware_slot=0
96142025-04-30T06:44:10.227ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "e6c1ab28-724c-423c-9bee-b7827c4d4da2", "content-length": "109", "date": "Wed, 30 Apr 2025 06:44:09 GMT"} })
96152025-04-30T06:44:10.227ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0/component/sp/caboose?firmware_slot=1
96162025-04-30T06:44:10.234ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "bf8f0f5f-425c-4ad9-914e-0493a20159b2", "content-length": "109", "date": "Wed, 30 Apr 2025 06:44:09 GMT"} })
96172025-04-30T06:44:10.236ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0/component/rot/caboose?firmware_slot=0
96182025-04-30T06:44:10.245ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "789b617e-7471-4bcb-8780-3ce11967571d", "content-length": "169", "date": "Wed, 30 Apr 2025 06:44:09 GMT"} })
96192025-04-30T06:44:10.245ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0/component/rot/caboose?firmware_slot=1
96202025-04-30T06:44:10.251ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "cc77d1d2-33ba-4ab7-b58e-5be324ebd7d8", "content-length": "169", "date": "Wed, 30 Apr 2025 06:44:09 GMT"} })
96212025-04-30T06:44:10.251ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0/component/stage0/caboose?firmware_slot=0
96222025-04-30T06:44:10.254ZDEBGtest_utilization: attempt to fetch latest inventory collection result = Ok("found")
96232025-04-30T06:44:10.258ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "daa15465-e4ea-4cce-8ccb-41270fc24345", "content-length": "178", "date": "Wed, 30 Apr 2025 06:44:09 GMT"} })
96242025-04-30T06:44:10.258ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0/component/stage0/caboose?firmware_slot=1
96252025-04-30T06:44:10.261ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "8ec351f1-0ade-40f4-aa30-e61a0fbdd634", "content-length": "178", "date": "Wed, 30 Apr 2025 06:44:09 GMT"} })
96262025-04-30T06:44:10.261ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0/component/rot/cmpa
96272025-04-30T06:44:10.262ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "001ef577-096c-497c-9ced-6317b59987e4", "content-length": "702", "date": "Wed, 30 Apr 2025 06:44:09 GMT"} })
96282025-04-30T06:44:10.262ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0/component/rot/cfpa
96292025-04-30T06:44:10.264ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "6aefa8d4-dfd0-4f69-8c1a-5e081d14d7aa", "content-length": "727", "date": "Wed, 30 Apr 2025 06:44:09 GMT"} })
96302025-04-30T06:44:10.264ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0/component/rot/cfpa
96312025-04-30T06:44:10.266ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "04cce0d2-118a-40d6-81af-4c4fe898c494", "content-length": "729", "date": "Wed, 30 Apr 2025 06:44:09 GMT"} })
96322025-04-30T06:44:10.266ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0/component/rot/cfpa
96332025-04-30T06:44:10.268ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "eb679fb8-711a-445d-9806-9a690907c708", "content-length": "728", "date": "Wed, 30 Apr 2025 06:44:09 GMT"} })
96342025-04-30T06:44:10.268ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1
96352025-04-30T06:44:10.269ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1", status: 200, headers: {"content-type": "application/json", "x-request-id": "5f796cb3-b696-4ec5-9c68-bad90b1017d0", "content-length": "723", "date": "Wed, 30 Apr 2025 06:44:09 GMT"} })
96362025-04-30T06:44:10.270ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1/component/sp/caboose?firmware_slot=0
96372025-04-30T06:44:10.273ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "ff79198a-d5d1-4eb6-aff0-1b5c8ed35290", "content-length": "109", "date": "Wed, 30 Apr 2025 06:44:09 GMT"} })
96382025-04-30T06:44:10.274ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1/component/sp/caboose?firmware_slot=1
96392025-04-30T06:44:10.276ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "e241a8cc-af8e-4510-9b50-e349def7cc4e", "content-length": "109", "date": "Wed, 30 Apr 2025 06:44:09 GMT"} })
96402025-04-30T06:44:10.276ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1/component/rot/caboose?firmware_slot=0
96412025-04-30T06:44:10.279ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "9a4759c4-5830-4982-b860-1fea487f4a90", "content-length": "169", "date": "Wed, 30 Apr 2025 06:44:09 GMT"} })
96422025-04-30T06:44:10.279ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1/component/rot/caboose?firmware_slot=1
96432025-04-30T06:44:10.282ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "d3af9c92-df7f-4db0-93f3-9aa6a28c81d4", "content-length": "169", "date": "Wed, 30 Apr 2025 06:44:09 GMT"} })
96442025-04-30T06:44:10.282ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1/component/stage0/caboose?firmware_slot=0
96452025-04-30T06:44:10.285ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "d3cc9fff-a1e5-49d3-ad0e-6fa0c8128ee4", "content-length": "178", "date": "Wed, 30 Apr 2025 06:44:09 GMT"} })
96462025-04-30T06:44:10.285ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1/component/stage0/caboose?firmware_slot=1
96472025-04-30T06:44:10.288ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "7ae45c07-cada-4a07-aa35-0f44c212f4e2", "content-length": "178", "date": "Wed, 30 Apr 2025 06:44:09 GMT"} })
96482025-04-30T06:44:10.288ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1/component/rot/cmpa
96492025-04-30T06:44:10.289ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "153378d8-6014-4c47-af4f-f032cadbed77", "content-length": "702", "date": "Wed, 30 Apr 2025 06:44:09 GMT"} })
96502025-04-30T06:44:10.289ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1/component/rot/cfpa
96512025-04-30T06:44:10.291ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "91e628a9-586d-4d60-a556-66531bd50c95", "content-length": "727", "date": "Wed, 30 Apr 2025 06:44:09 GMT"} })
96522025-04-30T06:44:10.291ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1/component/rot/cfpa
96532025-04-30T06:44:10.294ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "024b7f23-98db-4568-9981-3c0cca681304", "content-length": "729", "date": "Wed, 30 Apr 2025 06:44:09 GMT"} })
96542025-04-30T06:44:10.295ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1/component/rot/cfpa
96552025-04-30T06:44:10.296ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "c2767fc3-2259-4794-a3fd-d066124634c7", "content-length": "728", "date": "Wed, 30 Apr 2025 06:44:09 GMT"} })
96562025-04-30T06:44:10.296ZTRACtest_utilization (BackgroundTasks): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet
96572025-04-30T06:44:10.297ZTRACtest_utilization (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
96582025-04-30T06:44:10.297ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
96592025-04-30T06:44:10.297ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
96602025-04-30T06:44:10.298ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
96612025-04-30T06:44:10.304ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
96622025-04-30T06:44:10.315ZDEBGtest_utilization (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet result = Ok(())
96632025-04-30T06:44:10.315ZTRACtest_utilization (BackgroundTasks): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet
96642025-04-30T06:44:10.315ZTRACtest_utilization (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
96652025-04-30T06:44:10.315ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
96662025-04-30T06:44:10.315ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
96672025-04-30T06:44:10.315ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
96682025-04-30T06:44:10.316ZTRACtest_utilization (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory
96692025-04-30T06:44:10.316ZTRACtest_utilization (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
96702025-04-30T06:44:10.316ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
96712025-04-30T06:44:10.316ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
96722025-04-30T06:44:10.316ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
96732025-04-30T06:44:10.319ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
96742025-04-30T06:44:10.325ZDEBGtest_utilization (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet result = Ok(())
96752025-04-30T06:44:10.325ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
96762025-04-30T06:44:10.325ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
96772025-04-30T06:44:10.326ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
96782025-04-30T06:44:10.326ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
96792025-04-30T06:44:10.331ZDEBGtest_utilization (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory result = Ok(())
96802025-04-30T06:44:10.331ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
96812025-04-30T06:44:10.331ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
96822025-04-30T06:44:10.333ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
96832025-04-30T06:44:10.335ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
96842025-04-30T06:44:10.335ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
96852025-04-30T06:44:10.336ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
96862025-04-30T06:44:10.422ZDEBGtest_utilization (BackgroundTasks): begin collection from Sled Agent background_task = inventory_collection sled_agent_url = http://[::1]:39365
96872025-04-30T06:44:10.422ZDEBGtest_utilization (BackgroundTasks): client request SledAgent = http://[::1]:39365 background_task = inventory_collection body = None method = GET uri = http://[::1]:39365/inventory
96882025-04-30T06:44:10.423ZINFOtest_utilization (dropshot): accepted connection local_addr = [::1]:39365 remote_addr = [::1]:44849 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
96892025-04-30T06:44:10.423ZDEBGtest_utilization (results-sink): inserting 1 samples into database collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
96902025-04-30T06:44:10.423ZDEBGtest_utilization (results-sink-cluster): inserting 1 samples into database collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
96912025-04-30T06:44:10.424ZDEBGtest_utilization (clickhouse-client): unrolling 1 total samples collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 4d1485f0-110e-4161-9a33-c4610d68be5d
96922025-04-30T06:44:10.425ZTRACtest_utilization (clickhouse-client): inserting data collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 4d1485f0-110e-4161-9a33-c4610d68be5d n_columns = 4 n_rows = 2 sql = INSERT INTO oximeter.fields_string FORMAT Native
96932025-04-30T06:44:10.426ZTRACtest_utilization (dropshot): incoming request local_addr = [::1]:39365 method = GET remote_addr = [::1]:44849 req_id = 98e141f0-aa0d-41d5-8744-31665f295957 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /inventory
96942025-04-30T06:44:10.427ZINFOtest_utilization (dropshot): request completed latency_us = 833 local_addr = [::1]:39365 method = GET remote_addr = [::1]:44849 req_id = 98e141f0-aa0d-41d5-8744-31665f295957 response_code = 200 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /inventory
96952025-04-30T06:44:10.427ZDEBGtest_utilization (BackgroundTasks): client response SledAgent = http://[::1]:39365 background_task = inventory_collection result = Ok(Response { url: "http://[::1]:39365/inventory", status: 200, headers: {"content-type": "application/json", "x-request-id": "98e141f0-aa0d-41d5-8744-31665f295957", "content-length": "3141", "date": "Wed, 30 Apr 2025 06:44:09 GMT"} })
96962025-04-30T06:44:10.430ZDEBGtest_utilization (BackgroundTasks): begin collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
96972025-04-30T06:44:10.430ZDEBGtest_utilization (BackgroundTasks): end collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
96982025-04-30T06:44:10.430ZDEBGtest_utilization (BackgroundTasks): finished collection background_task = inventory_collection
96992025-04-30T06:44:10.433ZTRACtest_utilization (BackgroundTasks): authorize begin action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory
97002025-04-30T06:44:10.433ZTRACtest_utilization (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
97012025-04-30T06:44:10.434ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
97022025-04-30T06:44:10.434ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
97032025-04-30T06:44:10.435ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
97042025-04-30T06:44:10.440ZDEBGtest_utilization (clickhouse-client): inserted rows into table collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 4d1485f0-110e-4161-9a33-c4610d68be5d n_rows = 2 table_name = fields_string
97052025-04-30T06:44:10.441ZTRACtest_utilization (clickhouse-client): inserting data collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 4d1485f0-110e-4161-9a33-c4610d68be5d n_columns = 4 n_rows = 1 sql = INSERT INTO oximeter.measurements_i64 FORMAT Native
97062025-04-30T06:44:10.441ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
97072025-04-30T06:44:10.452ZDEBGtest_utilization (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory result = Ok(())
97082025-04-30T06:44:10.452ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
97092025-04-30T06:44:10.452ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
97102025-04-30T06:44:10.455ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
97112025-04-30T06:44:10.459ZDEBGtest_utilization (clickhouse-client): inserted rows into table collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 4d1485f0-110e-4161-9a33-c4610d68be5d n_rows = 1 table_name = measurements_i64
97122025-04-30T06:44:10.459ZTRACtest_utilization (results-sink): successfully inserted samples collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
97132025-04-30T06:44:10.506ZDEBGtest_utilization: attempt to fetch latest inventory collection result = Ok("found")
97142025-04-30T06:44:10.513ZWARNtest_utilization (datastore): Non-retryable transaction failure err = DatabaseError(UniqueViolation, "duplicate key value violates unique constraint \\"inv_caboose_pkey\\"") transaction = inventory_insert_collection
97152025-04-30T06:44:10.513ZWARNtest_utilization (BackgroundTasks): inventory collection failed background_task = inventory_collection error = failed to collect inventory: saving inventory to database: Internal Error: unexpected database error: duplicate key value violates unique constraint "inv_caboose_pkey"
97162025-04-30T06:44:10.513ZDEBGtest_utilization (BackgroundTasks): activation complete background_task = inventory_collection elapsed = 589.073571ms iteration = 20 status = {"error":"failed to collect inventory: saving inventory to database: Internal Error: unexpected database error: duplicate key value violates unique constraint \\"inv_caboose_pkey\\""}
97172025-04-30T06:44:10.513ZDEBGtest_utilization (BackgroundTasks): activating background_task = inventory_collection iteration = 21 reason = Signaled
97182025-04-30T06:44:10.513ZTRACtest_utilization (BackgroundTasks): authorize begin action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory
97192025-04-30T06:44:10.513ZTRACtest_utilization (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
97202025-04-30T06:44:10.513ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
97212025-04-30T06:44:10.513ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
97222025-04-30T06:44:10.514ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
97232025-04-30T06:44:10.521ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
97242025-04-30T06:44:10.525ZDEBGtest_utilization (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory result = Ok(())
97252025-04-30T06:44:10.525ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
97262025-04-30T06:44:10.525ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
97272025-04-30T06:44:10.525ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
97282025-04-30T06:44:10.526ZWARNtest_utilization (results-sink-cluster): failed to insert some results into metric DB collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 error = Failed to check out connection to database: No backends found for this service file = oximeter/collector/src/results_sink.rs:92
97292025-04-30T06:44:10.530ZDEBGtest_utilization (BackgroundTasks): inventory_prune_one: nothing eligible for removal (too few) background_task = inventory_collection candidates = [(13d8b8d9-6c6b-47a6-9190-952544edb75b, 0)]
97302025-04-30T06:44:10.530ZTRACtest_utilization (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal
97312025-04-30T06:44:10.530ZTRACtest_utilization (dns): buffer buffer = Length: 67 (0x43) bytes\n0000: c6 10 01 00 00 01 00 00 00 00 00 01 04 5f 6d 67 ............._mg\n0010: 73 04 5f 74 63 70 0d 63 6f 6e 74 72 6f 6c 2d 70 s._tcp.control-p\n0020: 6c 61 6e 65 05 6f 78 69 64 65 08 69 6e 74 65 72 lane.oxide.inter\n0030: 6e 61 6c 00 00 21 00 01 00 00 29 04 d0 00 00 00 nal..!....).....\n0040: 00 00 00 ... kind = dns peer_addr = [::1]:59255 req_id = 2711e52e-d263-4eb2-a2b9-e5464a26d8f3
97322025-04-30T06:44:10.531ZDEBGtest_utilization (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 50704,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_mgs._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_mgs._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 4,\n 95,\n 109,\n 103,\n 115,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:59255 req_id = 2711e52e-d263-4eb2-a2b9-e5464a26d8f3
97332025-04-30T06:44:10.532ZDEBGtest_utilization (store): query key key = _mgs._tcp
97342025-04-30T06:44:10.532ZDEBGtest_utilization (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
97352025-04-30T06:44:10.533ZDEBGtest_utilization (dns): dns response additional_records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal"), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] kind = dns peer_addr = [::1]:59255 query = LowerQuery { name: LowerName(Name("_mgs._tcp.control-plane.oxide.internal.")), original: Query { name: Name("_mgs._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN } } records = [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 48759, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal") })) }] req_id = 2711e52e-d263-4eb2-a2b9-e5464a26d8f3
97362025-04-30T06:44:10.533ZDEBGtest_utilization (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgs._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 48759, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }], valid_until: Instant { tv_sec: 4315, tv_nsec: 332673885 } })
97372025-04-30T06:44:10.533ZTRACtest_utilization (DnsResolver): lookup_service_targets: looking up SRV target
97382025-04-30T06:44:10.534ZTRACtest_utilization (dns): buffer buffer = Length: 108 (0x6c) bytes\n0000: a7 ae 01 00 00 01 00 00 00 00 00 01 2d 64 65 6e ............-den\n0010: 64 72 69 74 65 2d 62 36 64 36 35 33 34 31 2d 31 drite-b6d65341-1\n0020: 36 37 63 2d 34 31 64 66 2d 39 62 35 63 2d 34 31 67c-41df-9b5c-41\n0030: 63 64 65 64 39 39 63 32 32 39 04 68 6f 73 74 0d cded99c229.host.\n0040: 63 6f 6e 74 72 6f 6c 2d 70 6c 61 6e 65 05 6f 78 control-plane.ox\n0050: 69 64 65 08 69 6e 74 65 72 6e 61 6c 00 00 1c 00 ide.internal....\n0060: 01 00 00 29 04 d0 00 00 00 00 00 00 ...)........ kind = dns peer_addr = [::1]:49481 req_id = 90ac3ebc-cd9f-4e78-9e58-36538588b856
97392025-04-30T06:44:10.534ZDEBGtest_utilization (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 42926,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n query_type: AAAA,\n query_class: IN,\n },\n },\n original: [\n 45,\n 100,\n 101,\n 110,\n 100,\n 114,\n 105,\n 116,\n 101,\n 45,\n 98,\n 54,\n 100,\n 54,\n 53,\n 51,\n 52,\n 49,\n 45,\n 49,\n 54,\n 55,\n 99,\n 45,\n 52,\n 49,\n 100,\n 102,\n 45,\n 57,\n 98,\n 53,\n 99,\n 45,\n 52,\n 49,\n 99,\n 100,\n 101,\n 100,\n 57,\n 57,\n 99,\n 50,\n 50,\n 57,\n 4,\n 104,\n 111,\n 115,\n 116,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 28,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:49481 req_id = 90ac3ebc-cd9f-4e78-9e58-36538588b856
97402025-04-30T06:44:10.534ZDEBGtest_utilization (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
97412025-04-30T06:44:10.534ZDEBGtest_utilization (dns): dns response additional_records = [] kind = dns peer_addr = [::1]:49481 query = LowerQuery { name: LowerName(Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.")), original: Query { name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), query_type: AAAA, query_class: IN } } records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] req_id = 90ac3ebc-cd9f-4e78-9e58-36538588b856
97422025-04-30T06:44:10.598ZTRACtest_utilization (DnsResolver): lookup_all_socket_v6 srv dns_name = _clickhouse-admin-keeper._tcp.control-plane.oxide.internal
97432025-04-30T06:44:10.600ZTRACtest_utilization (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory
97442025-04-30T06:44:10.600ZTRACtest_utilization (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
97452025-04-30T06:44:10.600ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
97462025-04-30T06:44:10.600ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
97472025-04-30T06:44:10.602ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
97482025-04-30T06:44:10.603ZTRACtest_utilization (dns): buffer buffer = Length: 87 (0x57) bytes\n0000: 1c 85 01 00 00 01 00 00 00 00 00 01 18 5f 63 6c ............._cl\n0010: 69 63 6b 68 6f 75 73 65 2d 61 64 6d 69 6e 2d 6b ickhouse-admin-k\n0020: 65 65 70 65 72 04 5f 74 63 70 0d 63 6f 6e 74 72 eeper._tcp.contr\n0030: 6f 6c 2d 70 6c 61 6e 65 05 6f 78 69 64 65 08 69 ol-plane.oxide.i\n0040: 6e 74 65 72 6e 61 6c 00 00 21 00 01 00 00 29 04 nternal..!....).\n0050: d0 00 00 00 00 00 00 ....... kind = dns peer_addr = [::1]:63178 req_id = ead87336-f3a7-41ac-9714-24f9ae3e612e
97492025-04-30T06:44:10.603ZDEBGtest_utilization (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 7301,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 24,\n 95,\n 99,\n 108,\n 105,\n 99,\n 107,\n 104,\n 111,\n 117,\n 115,\n 101,\n 45,\n 97,\n 100,\n 109,\n 105,\n 110,\n 45,\n 107,\n 101,\n 101,\n 112,\n 101,\n 114,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:63178 req_id = ead87336-f3a7-41ac-9714-24f9ae3e612e
97502025-04-30T06:44:10.604ZDEBGtest_utilization (store): query key key = _clickhouse-admin-keeper._tcp
97512025-04-30T06:44:10.604ZERROtest_utilization (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 7301, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 1, }, query: WireQuery { query: LowerQuery { name: LowerName( Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), ), original: Query { name: Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN, }, }, original: [ 24, 95, 99, 108, 105, 99, 107, 104, 111, 117, 115, 101, 45, 97, 100, 109, 105, 110, 45, 107, 101, 101, 112, 101, 114, 4, 95, 116, 99, 112, 13, 99, 111, 110, 116, 114, 111, 108, 45, 112, 108, 97, 110, 101, 5, 111, 120, 105, 100, 101, 8, 105, 110, 116, 101, 114, 110, 97, 108, 0, 0, 33, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: Some( Edns { rcode_high: 0, version: 0, dnssec_ok: false, max_payload: 1232, options: OPT { options: {}, }, }, ), } NXDOMAIN: no records found for name: "_clickhouse-admin-keeper._tcp.control-plane.oxide.internal." kind = dns peer_addr = [::1]:63178 req_id = ead87336-f3a7-41ac-9714-24f9ae3e612e
97522025-04-30T06:44:10.605ZDEBGtest_utilization (BackgroundTasks): begin collection background_task = inventory_collection
97532025-04-30T06:44:10.605ZDEBGtest_utilization (BackgroundTasks): begin collection from MGS background_task = inventory_collection mgs_url = http://[::1]:48759
97542025-04-30T06:44:10.605ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/ignition
97552025-04-30T06:44:10.606ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
97562025-04-30T06:44:10.618ZDEBGtest_utilization (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory result = Ok(())
97572025-04-30T06:44:10.618ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
97582025-04-30T06:44:10.618ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
97592025-04-30T06:44:10.618ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
97602025-04-30T06:44:10.623ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
97612025-04-30T06:44:10.623ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
97622025-04-30T06:44:10.625ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
97632025-04-30T06:44:10.630ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/ignition", status: 200, headers: {"content-type": "application/json", "x-request-id": "d2d6f362-c7fc-44cf-9002-32be78609ab6", "content-length": "835", "date": "Wed, 30 Apr 2025 06:44:09 GMT"} })
97642025-04-30T06:44:10.630ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0
97652025-04-30T06:44:10.648ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "e1834c99-5efb-41a7-b16f-27530e8ecc37", "content-length": "734", "date": "Wed, 30 Apr 2025 06:44:09 GMT"} })
97662025-04-30T06:44:10.648ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0/component/sp/caboose?firmware_slot=0
97672025-04-30T06:44:10.661ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "365c89f2-6b0a-4e70-a42a-61fca766fb76", "content-length": "111", "date": "Wed, 30 Apr 2025 06:44:09 GMT"} })
97682025-04-30T06:44:10.661ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0/component/sp/caboose?firmware_slot=1
97692025-04-30T06:44:10.667ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "f5944ac3-6479-4a71-b53c-4a8a8012b80a", "content-length": "111", "date": "Wed, 30 Apr 2025 06:44:09 GMT"} })
97702025-04-30T06:44:10.668ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0/component/rot/caboose?firmware_slot=0
97712025-04-30T06:44:10.681ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "da225aad-be4c-448e-83c8-80acf25f340a", "content-length": "170", "date": "Wed, 30 Apr 2025 06:44:10 GMT"} })
97722025-04-30T06:44:10.681ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0/component/rot/caboose?firmware_slot=1
97732025-04-30T06:44:10.686ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "30251786-a1ad-488c-b370-b960aee65d3e", "content-length": "170", "date": "Wed, 30 Apr 2025 06:44:10 GMT"} })
97742025-04-30T06:44:10.686ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0/component/stage0/caboose?firmware_slot=0
97752025-04-30T06:44:10.692ZDEBGtest_utilization: attempt to fetch latest inventory collection result = Ok("found")
97762025-04-30T06:44:10.701ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "d6a3a7a5-0c89-4452-bdf1-7e5b1716da66", "content-length": "179", "date": "Wed, 30 Apr 2025 06:44:10 GMT"} })
97772025-04-30T06:44:10.701ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0/component/stage0/caboose?firmware_slot=1
97782025-04-30T06:44:10.705ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "1e45010b-05f4-4ae9-b56d-73f4bef843a8", "content-length": "179", "date": "Wed, 30 Apr 2025 06:44:10 GMT"} })
97792025-04-30T06:44:10.705ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0/component/rot/cmpa
97802025-04-30T06:44:10.706ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "76afc365-a912-43a8-bb44-921438f82b71", "content-length": "702", "date": "Wed, 30 Apr 2025 06:44:10 GMT"} })
97812025-04-30T06:44:10.707ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0/component/rot/cfpa
97822025-04-30T06:44:10.708ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "dd94f04e-effd-4082-ba3e-ec0f5ffedbb9", "content-length": "727", "date": "Wed, 30 Apr 2025 06:44:10 GMT"} })
97832025-04-30T06:44:10.708ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0/component/rot/cfpa
97842025-04-30T06:44:10.711ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "fb16b109-6a7a-44fc-bedd-f8ebaa214b11", "content-length": "729", "date": "Wed, 30 Apr 2025 06:44:10 GMT"} })
97852025-04-30T06:44:10.711ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0/component/rot/cfpa
97862025-04-30T06:44:10.712ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "6d567251-b53a-4189-bc48-aba5b199ef9c", "content-length": "728", "date": "Wed, 30 Apr 2025 06:44:10 GMT"} })
97872025-04-30T06:44:10.713ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1
97882025-04-30T06:44:10.714ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1", status: 200, headers: {"content-type": "application/json", "x-request-id": "d702c72c-b06c-4a66-bb05-2503be51a378", "content-length": "734", "date": "Wed, 30 Apr 2025 06:44:10 GMT"} })
97892025-04-30T06:44:10.715ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1/component/sp/caboose?firmware_slot=0
97902025-04-30T06:44:10.718ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "9feb76e9-88f7-4982-ac4c-880ef75ec2bf", "content-length": "111", "date": "Wed, 30 Apr 2025 06:44:10 GMT"} })
97912025-04-30T06:44:10.718ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1/component/sp/caboose?firmware_slot=1
97922025-04-30T06:44:10.727ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "19c5a10f-a0ae-4f91-9db7-43993d374f3f", "content-length": "111", "date": "Wed, 30 Apr 2025 06:44:10 GMT"} })
97932025-04-30T06:44:10.727ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1/component/rot/caboose?firmware_slot=0
97942025-04-30T06:44:10.731ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "c3ab97e8-e029-4b7d-9f84-f267b1b5ac26", "content-length": "170", "date": "Wed, 30 Apr 2025 06:44:10 GMT"} })
97952025-04-30T06:44:10.731ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1/component/rot/caboose?firmware_slot=1
97962025-04-30T06:44:10.733ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "a43d7d69-8ab9-4315-bc1f-10b93471e563", "content-length": "170", "date": "Wed, 30 Apr 2025 06:44:10 GMT"} })
97972025-04-30T06:44:10.734ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1/component/stage0/caboose?firmware_slot=0
97982025-04-30T06:44:10.736ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "16c869e9-5df7-44bf-b94e-41afafc39100", "content-length": "179", "date": "Wed, 30 Apr 2025 06:44:10 GMT"} })
97992025-04-30T06:44:10.736ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1/component/stage0/caboose?firmware_slot=1
98002025-04-30T06:44:10.738ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "4596bd93-9fad-4e68-80b2-4bd0466aaafa", "content-length": "179", "date": "Wed, 30 Apr 2025 06:44:10 GMT"} })
98012025-04-30T06:44:10.738ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1/component/rot/cmpa
98022025-04-30T06:44:10.739ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "0fd54955-5a01-497e-85dd-9b91307b5708", "content-length": "702", "date": "Wed, 30 Apr 2025 06:44:10 GMT"} })
98032025-04-30T06:44:10.740ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1/component/rot/cfpa
98042025-04-30T06:44:10.740ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "bcb33a96-4da3-4f22-a12b-4658032f5732", "content-length": "727", "date": "Wed, 30 Apr 2025 06:44:10 GMT"} })
98052025-04-30T06:44:10.741ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1/component/rot/cfpa
98062025-04-30T06:44:10.742ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "582cc4a5-b960-4f95-91bf-ab2402f109fc", "content-length": "729", "date": "Wed, 30 Apr 2025 06:44:10 GMT"} })
98072025-04-30T06:44:10.742ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1/component/rot/cfpa
98082025-04-30T06:44:10.742ZTRACtest_utilization (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory
98092025-04-30T06:44:10.742ZTRACtest_utilization (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
98102025-04-30T06:44:10.742ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
98112025-04-30T06:44:10.742ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
98122025-04-30T06:44:10.744ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
98132025-04-30T06:44:10.745ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "4f4302a1-9c6f-4348-a524-2ec9399645eb", "content-length": "728", "date": "Wed, 30 Apr 2025 06:44:10 GMT"} })
98142025-04-30T06:44:10.745ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0
98152025-04-30T06:44:10.747ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
98162025-04-30T06:44:10.765ZDEBGtest_utilization (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory result = Ok(())
98172025-04-30T06:44:10.765ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
98182025-04-30T06:44:10.765ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
98192025-04-30T06:44:10.767ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
98202025-04-30T06:44:10.769ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "7e8bc758-2f82-4984-a7fe-bda91b973c2a", "content-length": "723", "date": "Wed, 30 Apr 2025 06:44:10 GMT"} })
98212025-04-30T06:44:10.769ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0/component/sp/caboose?firmware_slot=0
98222025-04-30T06:44:10.773ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "7e91f99f-5b36-4aaa-9d0e-6bbb16966379", "content-length": "109", "date": "Wed, 30 Apr 2025 06:44:10 GMT"} })
98232025-04-30T06:44:10.776ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0/component/sp/caboose?firmware_slot=1
98242025-04-30T06:44:10.779ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
98252025-04-30T06:44:10.779ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
98262025-04-30T06:44:10.779ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
98272025-04-30T06:44:10.782ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "d1d0e50c-2221-4ef7-b7e3-334dcfa850c3", "content-length": "109", "date": "Wed, 30 Apr 2025 06:44:10 GMT"} })
98282025-04-30T06:44:10.782ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0/component/rot/caboose?firmware_slot=0
98292025-04-30T06:44:10.784ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "eb5a8988-cc99-4381-9099-4553197609b2", "content-length": "169", "date": "Wed, 30 Apr 2025 06:44:10 GMT"} })
98302025-04-30T06:44:10.784ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0/component/rot/caboose?firmware_slot=1
98312025-04-30T06:44:10.787ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "8e1cae1d-223c-45d3-b908-90a36b50f513", "content-length": "169", "date": "Wed, 30 Apr 2025 06:44:10 GMT"} })
98322025-04-30T06:44:10.787ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0/component/stage0/caboose?firmware_slot=0
98332025-04-30T06:44:10.792ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "42b06902-3196-4801-b198-58fe51b53f9b", "content-length": "178", "date": "Wed, 30 Apr 2025 06:44:10 GMT"} })
98342025-04-30T06:44:10.792ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0/component/stage0/caboose?firmware_slot=1
98352025-04-30T06:44:10.799ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "a466fe82-e4b1-4fc6-b9d6-2f3c3d448aa7", "content-length": "178", "date": "Wed, 30 Apr 2025 06:44:10 GMT"} })
98362025-04-30T06:44:10.799ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0/component/rot/cmpa
98372025-04-30T06:44:10.809ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "8fc0630a-1447-4dc4-b247-33c92cdc3d2b", "content-length": "702", "date": "Wed, 30 Apr 2025 06:44:10 GMT"} })
98382025-04-30T06:44:10.809ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0/component/rot/cfpa
98392025-04-30T06:44:10.811ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "249e0665-512c-4ab8-a962-b18499a5bb8f", "content-length": "727", "date": "Wed, 30 Apr 2025 06:44:10 GMT"} })
98402025-04-30T06:44:10.811ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0/component/rot/cfpa
98412025-04-30T06:44:10.812ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "4c00a063-60fa-4376-9802-4e6165a123b0", "content-length": "729", "date": "Wed, 30 Apr 2025 06:44:10 GMT"} })
98422025-04-30T06:44:10.813ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0/component/rot/cfpa
98432025-04-30T06:44:10.815ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "6172d9b1-a450-4f33-87fb-5b48c6232012", "content-length": "728", "date": "Wed, 30 Apr 2025 06:44:10 GMT"} })
98442025-04-30T06:44:10.815ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1
98452025-04-30T06:44:10.818ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1", status: 200, headers: {"content-type": "application/json", "x-request-id": "84215144-0935-4c02-b301-8df3ad96c654", "content-length": "723", "date": "Wed, 30 Apr 2025 06:44:10 GMT"} })
98462025-04-30T06:44:10.818ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1/component/sp/caboose?firmware_slot=0
98472025-04-30T06:44:10.824ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "dcb9d66b-495c-4926-9b2b-b9b56f54f402", "content-length": "109", "date": "Wed, 30 Apr 2025 06:44:10 GMT"} })
98482025-04-30T06:44:10.824ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1/component/sp/caboose?firmware_slot=1
98492025-04-30T06:44:10.841ZDEBGtest_utilization: attempt to fetch latest inventory collection result = Ok("found")
98502025-04-30T06:44:10.849ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "32107a54-7365-440a-bd44-9ac9f9130396", "content-length": "109", "date": "Wed, 30 Apr 2025 06:44:10 GMT"} })
98512025-04-30T06:44:10.849ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1/component/rot/caboose?firmware_slot=0
98522025-04-30T06:44:10.851ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "06f256cf-5420-4963-a9d3-c47b9d43d872", "content-length": "169", "date": "Wed, 30 Apr 2025 06:44:10 GMT"} })
98532025-04-30T06:44:10.852ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1/component/rot/caboose?firmware_slot=1
98542025-04-30T06:44:10.854ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "dfac977d-3d81-4082-8ad1-e514876e9dbb", "content-length": "169", "date": "Wed, 30 Apr 2025 06:44:10 GMT"} })
98552025-04-30T06:44:10.854ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1/component/stage0/caboose?firmware_slot=0
98562025-04-30T06:44:10.856ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "b30e9160-61a5-4aa3-b14d-9cb72ef8db5d", "content-length": "178", "date": "Wed, 30 Apr 2025 06:44:10 GMT"} })
98572025-04-30T06:44:10.856ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1/component/stage0/caboose?firmware_slot=1
98582025-04-30T06:44:10.858ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "b10ca64f-aeb4-4900-8108-ae433b7af0c6", "content-length": "178", "date": "Wed, 30 Apr 2025 06:44:10 GMT"} })
98592025-04-30T06:44:10.859ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1/component/rot/cmpa
98602025-04-30T06:44:10.859ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "014e8e46-b953-4f3a-8d9c-22bc07d300ba", "content-length": "702", "date": "Wed, 30 Apr 2025 06:44:10 GMT"} })
98612025-04-30T06:44:10.860ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1/component/rot/cfpa
98622025-04-30T06:44:10.861ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "ceae7d0e-94ad-4c0d-b1a6-baddd2b8bb97", "content-length": "727", "date": "Wed, 30 Apr 2025 06:44:10 GMT"} })
98632025-04-30T06:44:10.861ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1/component/rot/cfpa
98642025-04-30T06:44:10.864ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "01203df5-acc5-43e2-a9e8-b2236f78c291", "content-length": "729", "date": "Wed, 30 Apr 2025 06:44:10 GMT"} })
98652025-04-30T06:44:10.864ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1/component/rot/cfpa
98662025-04-30T06:44:10.865ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "a1b52248-2ee3-4527-b5f8-47b9513788ca", "content-length": "728", "date": "Wed, 30 Apr 2025 06:44:10 GMT"} })
98672025-04-30T06:44:10.865ZTRACtest_utilization (BackgroundTasks): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet
98682025-04-30T06:44:10.865ZTRACtest_utilization (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
98692025-04-30T06:44:10.865ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
98702025-04-30T06:44:10.865ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
98712025-04-30T06:44:10.866ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
98722025-04-30T06:44:10.870ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
98732025-04-30T06:44:10.874ZDEBGtest_utilization (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet result = Ok(())
98742025-04-30T06:44:10.874ZTRACtest_utilization (BackgroundTasks): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet
98752025-04-30T06:44:10.874ZTRACtest_utilization (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
98762025-04-30T06:44:10.874ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
98772025-04-30T06:44:10.874ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
98782025-04-30T06:44:10.875ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
98792025-04-30T06:44:10.880ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
98802025-04-30T06:44:10.886ZDEBGtest_utilization (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet result = Ok(())
98812025-04-30T06:44:10.886ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
98822025-04-30T06:44:10.886ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
98832025-04-30T06:44:10.887ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
98842025-04-30T06:44:10.893ZTRACtest_utilization (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory
98852025-04-30T06:44:10.893ZTRACtest_utilization (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
98862025-04-30T06:44:10.893ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
98872025-04-30T06:44:10.893ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
98882025-04-30T06:44:10.895ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
98892025-04-30T06:44:10.947ZDEBGtest_utilization (BackgroundTasks): begin collection from Sled Agent background_task = inventory_collection sled_agent_url = http://[::1]:39365
98902025-04-30T06:44:10.947ZDEBGtest_utilization (BackgroundTasks): client request SledAgent = http://[::1]:39365 background_task = inventory_collection body = None method = GET uri = http://[::1]:39365/inventory
98912025-04-30T06:44:10.948ZINFOtest_utilization (dropshot): accepted connection local_addr = [::1]:39365 remote_addr = [::1]:59666 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
98922025-04-30T06:44:10.949ZTRACtest_utilization (dropshot): incoming request local_addr = [::1]:39365 method = GET remote_addr = [::1]:59666 req_id = 22b8ce55-818a-4d10-bad2-3345347dac3a sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /inventory
98932025-04-30T06:44:10.949ZINFOtest_utilization (dropshot): request completed latency_us = 664 local_addr = [::1]:39365 method = GET remote_addr = [::1]:59666 req_id = 22b8ce55-818a-4d10-bad2-3345347dac3a response_code = 200 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /inventory
98942025-04-30T06:44:10.950ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
98952025-04-30T06:44:10.957ZDEBGtest_utilization (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory result = Ok(())
98962025-04-30T06:44:10.957ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
98972025-04-30T06:44:10.957ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
98982025-04-30T06:44:10.958ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
98992025-04-30T06:44:10.959ZDEBGtest_utilization (BackgroundTasks): client response SledAgent = http://[::1]:39365 background_task = inventory_collection result = Ok(Response { url: "http://[::1]:39365/inventory", status: 200, headers: {"content-type": "application/json", "x-request-id": "22b8ce55-818a-4d10-bad2-3345347dac3a", "content-length": "3141", "date": "Wed, 30 Apr 2025 06:44:10 GMT"} })
99002025-04-30T06:44:10.962ZDEBGtest_utilization (BackgroundTasks): begin collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
99012025-04-30T06:44:10.962ZDEBGtest_utilization (BackgroundTasks): end collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
99022025-04-30T06:44:10.962ZDEBGtest_utilization (BackgroundTasks): finished collection background_task = inventory_collection
99032025-04-30T06:44:10.966ZTRACtest_utilization (BackgroundTasks): authorize begin action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory
99042025-04-30T06:44:10.966ZTRACtest_utilization (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
99052025-04-30T06:44:10.966ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
99062025-04-30T06:44:10.967ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
99072025-04-30T06:44:10.967ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
99082025-04-30T06:44:10.972ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
99092025-04-30T06:44:10.972ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
99102025-04-30T06:44:10.973ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
99112025-04-30T06:44:10.975ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
99122025-04-30T06:44:10.981ZDEBGtest_utilization (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory result = Ok(())
99132025-04-30T06:44:10.981ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
99142025-04-30T06:44:10.982ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
99152025-04-30T06:44:10.982ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
99162025-04-30T06:44:10.999ZDEBGtest_utilization (collection-task): sent timer-based collection request to the collection task collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
99172025-04-30T06:44:11.003ZDEBGtest_utilization (collection-task): collecting from producer collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
99182025-04-30T06:44:11.007ZTRACtest_utilization (collection-task): sent collection request to producer collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
99192025-04-30T06:44:11.010ZDEBGtest_utilization (collection-task): collected results from producer collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 n_results = 1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
99202025-04-30T06:44:11.010ZTRACtest_utilization (collection-task): forwarded results to main collection loop collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
99212025-04-30T06:44:11.010ZTRACtest_utilization (collection-task): top of inner collection loop, waiting for next request collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
99222025-04-30T06:44:11.045ZDEBGtest_utilization: attempt to fetch latest inventory collection result = Ok("found")
99232025-04-30T06:44:11.050ZWARNtest_utilization (datastore): Non-retryable transaction failure err = DatabaseError(UniqueViolation, "duplicate key value violates unique constraint \\"inv_caboose_pkey\\"") transaction = inventory_insert_collection
99242025-04-30T06:44:11.050ZWARNtest_utilization (BackgroundTasks): inventory collection failed background_task = inventory_collection error = failed to collect inventory: saving inventory to database: Internal Error: unexpected database error: duplicate key value violates unique constraint "inv_caboose_pkey"
99252025-04-30T06:44:11.051ZDEBGtest_utilization (BackgroundTasks): activation complete background_task = inventory_collection elapsed = 537.251167ms iteration = 21 status = {"error":"failed to collect inventory: saving inventory to database: Internal Error: unexpected database error: duplicate key value violates unique constraint \\"inv_caboose_pkey\\""}
99262025-04-30T06:44:11.051ZDEBGtest_utilization (BackgroundTasks): activating background_task = inventory_collection iteration = 22 reason = Signaled
99272025-04-30T06:44:11.051ZTRACtest_utilization (BackgroundTasks): authorize begin action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory
99282025-04-30T06:44:11.051ZTRACtest_utilization (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
99292025-04-30T06:44:11.051ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
99302025-04-30T06:44:11.051ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
99312025-04-30T06:44:11.053ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
99322025-04-30T06:44:11.059ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
99332025-04-30T06:44:11.068ZDEBGtest_utilization (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory result = Ok(())
99342025-04-30T06:44:11.068ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
99352025-04-30T06:44:11.068ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
99362025-04-30T06:44:11.069ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
99372025-04-30T06:44:11.074ZDEBGtest_utilization (BackgroundTasks): inventory_prune_one: nothing eligible for removal (too few) background_task = inventory_collection candidates = [(13d8b8d9-6c6b-47a6-9190-952544edb75b, 0)]
99382025-04-30T06:44:11.074ZTRACtest_utilization (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal
99392025-04-30T06:44:11.076ZTRACtest_utilization (dns): buffer buffer = Length: 67 (0x43) bytes\n0000: 9a 62 01 00 00 01 00 00 00 00 00 01 04 5f 6d 67 .b..........._mg\n0010: 73 04 5f 74 63 70 0d 63 6f 6e 74 72 6f 6c 2d 70 s._tcp.control-p\n0020: 6c 61 6e 65 05 6f 78 69 64 65 08 69 6e 74 65 72 lane.oxide.inter\n0030: 6e 61 6c 00 00 21 00 01 00 00 29 04 d0 00 00 00 nal..!....).....\n0040: 00 00 00 ... kind = dns peer_addr = [::1]:51348 req_id = a08ade82-a136-40d2-807d-3ef98787d8ec
99402025-04-30T06:44:11.076ZDEBGtest_utilization (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 39522,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_mgs._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_mgs._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 4,\n 95,\n 109,\n 103,\n 115,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:51348 req_id = a08ade82-a136-40d2-807d-3ef98787d8ec
99412025-04-30T06:44:11.077ZDEBGtest_utilization (store): query key key = _mgs._tcp
99422025-04-30T06:44:11.077ZDEBGtest_utilization (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
99432025-04-30T06:44:11.077ZDEBGtest_utilization (dns): dns response additional_records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal"), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] kind = dns peer_addr = [::1]:51348 query = LowerQuery { name: LowerName(Name("_mgs._tcp.control-plane.oxide.internal.")), original: Query { name: Name("_mgs._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN } } records = [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 48759, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal") })) }] req_id = a08ade82-a136-40d2-807d-3ef98787d8ec
99442025-04-30T06:44:11.078ZDEBGtest_utilization (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgs._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 48759, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }], valid_until: Instant { tv_sec: 4315, tv_nsec: 877041141 } })
99452025-04-30T06:44:11.078ZTRACtest_utilization (DnsResolver): lookup_service_targets: looking up SRV target
99462025-04-30T06:44:11.079ZTRACtest_utilization (dns): buffer buffer = Length: 108 (0x6c) bytes\n0000: a7 2b 01 00 00 01 00 00 00 00 00 01 2d 64 65 6e .+..........-den\n0010: 64 72 69 74 65 2d 62 36 64 36 35 33 34 31 2d 31 drite-b6d65341-1\n0020: 36 37 63 2d 34 31 64 66 2d 39 62 35 63 2d 34 31 67c-41df-9b5c-41\n0030: 63 64 65 64 39 39 63 32 32 39 04 68 6f 73 74 0d cded99c229.host.\n0040: 63 6f 6e 74 72 6f 6c 2d 70 6c 61 6e 65 05 6f 78 control-plane.ox\n0050: 69 64 65 08 69 6e 74 65 72 6e 61 6c 00 00 1c 00 ide.internal....\n0060: 01 00 00 29 04 d0 00 00 00 00 00 00 ...)........ kind = dns peer_addr = [::1]:52008 req_id = 64cfe6a8-0770-4192-a1a1-574029056067
99472025-04-30T06:44:11.079ZDEBGtest_utilization (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 42795,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n query_type: AAAA,\n query_class: IN,\n },\n },\n original: [\n 45,\n 100,\n 101,\n 110,\n 100,\n 114,\n 105,\n 116,\n 101,\n 45,\n 98,\n 54,\n 100,\n 54,\n 53,\n 51,\n 52,\n 49,\n 45,\n 49,\n 54,\n 55,\n 99,\n 45,\n 52,\n 49,\n 100,\n 102,\n 45,\n 57,\n 98,\n 53,\n 99,\n 45,\n 52,\n 49,\n 99,\n 100,\n 101,\n 100,\n 57,\n 57,\n 99,\n 50,\n 50,\n 57,\n 4,\n 104,\n 111,\n 115,\n 116,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 28,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:52008 req_id = 64cfe6a8-0770-4192-a1a1-574029056067
99482025-04-30T06:44:11.079ZDEBGtest_utilization (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
99492025-04-30T06:44:11.079ZDEBGtest_utilization (dns): dns response additional_records = [] kind = dns peer_addr = [::1]:52008 query = LowerQuery { name: LowerName(Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.")), original: Query { name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), query_type: AAAA, query_class: IN } } records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] req_id = 64cfe6a8-0770-4192-a1a1-574029056067
99502025-04-30T06:44:11.129ZTRACtest_utilization (DnsResolver): lookup_all_socket_v6 srv dns_name = _clickhouse-admin-keeper._tcp.control-plane.oxide.internal
99512025-04-30T06:44:11.129ZTRACtest_utilization (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory
99522025-04-30T06:44:11.130ZTRACtest_utilization (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
99532025-04-30T06:44:11.130ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
99542025-04-30T06:44:11.130ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
99552025-04-30T06:44:11.130ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
99562025-04-30T06:44:11.131ZTRACtest_utilization (dns): buffer buffer = Length: 87 (0x57) bytes\n0000: ec b1 01 00 00 01 00 00 00 00 00 01 18 5f 63 6c ............._cl\n0010: 69 63 6b 68 6f 75 73 65 2d 61 64 6d 69 6e 2d 6b ickhouse-admin-k\n0020: 65 65 70 65 72 04 5f 74 63 70 0d 63 6f 6e 74 72 eeper._tcp.contr\n0030: 6f 6c 2d 70 6c 61 6e 65 05 6f 78 69 64 65 08 69 ol-plane.oxide.i\n0040: 6e 74 65 72 6e 61 6c 00 00 21 00 01 00 00 29 04 nternal..!....).\n0050: d0 00 00 00 00 00 00 ....... kind = dns peer_addr = [::1]:52702 req_id = 711646c3-a6f9-4873-9f6c-75e20c349ae0
99572025-04-30T06:44:11.131ZDEBGtest_utilization (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 60593,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 24,\n 95,\n 99,\n 108,\n 105,\n 99,\n 107,\n 104,\n 111,\n 117,\n 115,\n 101,\n 45,\n 97,\n 100,\n 109,\n 105,\n 110,\n 45,\n 107,\n 101,\n 101,\n 112,\n 101,\n 114,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:52702 req_id = 711646c3-a6f9-4873-9f6c-75e20c349ae0
99582025-04-30T06:44:11.132ZDEBGtest_utilization (store): query key key = _clickhouse-admin-keeper._tcp
99592025-04-30T06:44:11.132ZERROtest_utilization (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 60593, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 1, }, query: WireQuery { query: LowerQuery { name: LowerName( Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), ), original: Query { name: Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN, }, }, original: [ 24, 95, 99, 108, 105, 99, 107, 104, 111, 117, 115, 101, 45, 97, 100, 109, 105, 110, 45, 107, 101, 101, 112, 101, 114, 4, 95, 116, 99, 112, 13, 99, 111, 110, 116, 114, 111, 108, 45, 112, 108, 97, 110, 101, 5, 111, 120, 105, 100, 101, 8, 105, 110, 116, 101, 114, 110, 97, 108, 0, 0, 33, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: Some( Edns { rcode_high: 0, version: 0, dnssec_ok: false, max_payload: 1232, options: OPT { options: {}, }, }, ), } NXDOMAIN: no records found for name: "_clickhouse-admin-keeper._tcp.control-plane.oxide.internal." kind = dns peer_addr = [::1]:52702 req_id = 711646c3-a6f9-4873-9f6c-75e20c349ae0
99602025-04-30T06:44:11.132ZDEBGtest_utilization (BackgroundTasks): begin collection background_task = inventory_collection
99612025-04-30T06:44:11.132ZDEBGtest_utilization (BackgroundTasks): begin collection from MGS background_task = inventory_collection mgs_url = http://[::1]:48759
99622025-04-30T06:44:11.132ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/ignition
99632025-04-30T06:44:11.134ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
99642025-04-30T06:44:11.139ZDEBGtest_utilization (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory result = Ok(())
99652025-04-30T06:44:11.139ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
99662025-04-30T06:44:11.139ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
99672025-04-30T06:44:11.140ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
99682025-04-30T06:44:11.142ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/ignition", status: 200, headers: {"content-type": "application/json", "x-request-id": "45ad8af9-3c87-43a0-a8ed-cfa780b379f4", "content-length": "835", "date": "Wed, 30 Apr 2025 06:44:10 GMT"} })
99692025-04-30T06:44:11.142ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0
99702025-04-30T06:44:11.143ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
99712025-04-30T06:44:11.143ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
99722025-04-30T06:44:11.143ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
99732025-04-30T06:44:11.145ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "497a307d-5216-491a-81e7-d3cc2b897565", "content-length": "734", "date": "Wed, 30 Apr 2025 06:44:10 GMT"} })
99742025-04-30T06:44:11.145ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0/component/sp/caboose?firmware_slot=0
99752025-04-30T06:44:11.149ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "a96b0ce9-0177-40f1-810f-e3ae3eff092e", "content-length": "111", "date": "Wed, 30 Apr 2025 06:44:10 GMT"} })
99762025-04-30T06:44:11.149ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0/component/sp/caboose?firmware_slot=1
99772025-04-30T06:44:11.151ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "66430f75-692f-4d82-95e6-c1efae82b208", "content-length": "111", "date": "Wed, 30 Apr 2025 06:44:10 GMT"} })
99782025-04-30T06:44:11.151ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0/component/rot/caboose?firmware_slot=0
99792025-04-30T06:44:11.156ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "385ca363-2315-4100-8a69-bbc4aee07389", "content-length": "170", "date": "Wed, 30 Apr 2025 06:44:10 GMT"} })
99802025-04-30T06:44:11.156ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0/component/rot/caboose?firmware_slot=1
99812025-04-30T06:44:11.160ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "d1d1fc2d-17b0-424e-8b42-deb5c8bcdb44", "content-length": "170", "date": "Wed, 30 Apr 2025 06:44:10 GMT"} })
99822025-04-30T06:44:11.160ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0/component/stage0/caboose?firmware_slot=0
99832025-04-30T06:44:11.164ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "5db0f794-8e81-4de9-a19f-fd797ef1f8f8", "content-length": "179", "date": "Wed, 30 Apr 2025 06:44:10 GMT"} })
99842025-04-30T06:44:11.164ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0/component/stage0/caboose?firmware_slot=1
99852025-04-30T06:44:11.169ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "be2af266-4e4b-4fd1-bf05-60d66fb9fefe", "content-length": "179", "date": "Wed, 30 Apr 2025 06:44:10 GMT"} })
99862025-04-30T06:44:11.169ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0/component/rot/cmpa
99872025-04-30T06:44:11.175ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "74339d98-6de6-4ca7-ad40-d8c8da5e1551", "content-length": "702", "date": "Wed, 30 Apr 2025 06:44:10 GMT"} })
99882025-04-30T06:44:11.175ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0/component/rot/cfpa
99892025-04-30T06:44:11.176ZDEBGtest_utilization: attempt to fetch latest inventory collection result = Ok("found")
99902025-04-30T06:44:11.179ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "5d31f6e1-afbf-4761-ac52-6e41652136f9", "content-length": "727", "date": "Wed, 30 Apr 2025 06:44:10 GMT"} })
99912025-04-30T06:44:11.180ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0/component/rot/cfpa
99922025-04-30T06:44:11.181ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "b3288377-0d36-4309-b595-3a92466615fc", "content-length": "729", "date": "Wed, 30 Apr 2025 06:44:10 GMT"} })
99932025-04-30T06:44:11.182ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0/component/rot/cfpa
99942025-04-30T06:44:11.186ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "86ad766d-885c-4360-9dca-9dc8533b61eb", "content-length": "728", "date": "Wed, 30 Apr 2025 06:44:10 GMT"} })
99952025-04-30T06:44:11.186ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1
99962025-04-30T06:44:11.189ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1", status: 200, headers: {"content-type": "application/json", "x-request-id": "72c5c385-a792-4598-b272-ec199431fda5", "content-length": "734", "date": "Wed, 30 Apr 2025 06:44:10 GMT"} })
99972025-04-30T06:44:11.191ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1/component/sp/caboose?firmware_slot=0
99982025-04-30T06:44:11.196ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "0e120c16-3728-4340-931b-93955329ed06", "content-length": "111", "date": "Wed, 30 Apr 2025 06:44:10 GMT"} })
99992025-04-30T06:44:11.196ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1/component/sp/caboose?firmware_slot=1
100002025-04-30T06:44:11.200ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "7b077787-fb48-418a-b3cc-d6255bc24ff1", "content-length": "111", "date": "Wed, 30 Apr 2025 06:44:10 GMT"} })
100012025-04-30T06:44:11.201ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1/component/rot/caboose?firmware_slot=0
100022025-04-30T06:44:11.204ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "99ce9442-76a4-42f5-a37d-bd3c13065037", "content-length": "170", "date": "Wed, 30 Apr 2025 06:44:10 GMT"} })
100032025-04-30T06:44:11.205ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1/component/rot/caboose?firmware_slot=1
100042025-04-30T06:44:11.209ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "a5d65a8e-cab9-4cc8-b166-2e9a8d4989e5", "content-length": "170", "date": "Wed, 30 Apr 2025 06:44:10 GMT"} })
100052025-04-30T06:44:11.209ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1/component/stage0/caboose?firmware_slot=0
100062025-04-30T06:44:11.211ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "d1fd4666-4e1e-4bef-9cd7-3178e51cca6c", "content-length": "179", "date": "Wed, 30 Apr 2025 06:44:10 GMT"} })
100072025-04-30T06:44:11.211ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1/component/stage0/caboose?firmware_slot=1
100082025-04-30T06:44:11.216ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "96bcacff-d0ab-40b1-b867-bd328af2e7f2", "content-length": "179", "date": "Wed, 30 Apr 2025 06:44:10 GMT"} })
100092025-04-30T06:44:11.217ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1/component/rot/cmpa
100102025-04-30T06:44:11.219ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "b044da17-2acb-4a0b-bd8a-515538dbc75c", "content-length": "702", "date": "Wed, 30 Apr 2025 06:44:10 GMT"} })
100112025-04-30T06:44:11.219ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1/component/rot/cfpa
100122025-04-30T06:44:11.222ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "99201b4f-d6ff-411d-a9b5-3bce1e7d073a", "content-length": "727", "date": "Wed, 30 Apr 2025 06:44:10 GMT"} })
100132025-04-30T06:44:11.223ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1/component/rot/cfpa
100142025-04-30T06:44:11.225ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "9aba7bb4-3e45-4d70-879c-6ac3bc33810e", "content-length": "729", "date": "Wed, 30 Apr 2025 06:44:10 GMT"} })
100152025-04-30T06:44:11.226ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1/component/rot/cfpa
100162025-04-30T06:44:11.227ZTRACtest_utilization (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory
100172025-04-30T06:44:11.227ZTRACtest_utilization (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
100182025-04-30T06:44:11.227ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
100192025-04-30T06:44:11.227ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
100202025-04-30T06:44:11.228ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
100212025-04-30T06:44:11.228ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "9fb0f5af-e6ce-429b-9957-135e29b773a1", "content-length": "728", "date": "Wed, 30 Apr 2025 06:44:10 GMT"} })
100222025-04-30T06:44:11.228ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0
100232025-04-30T06:44:11.231ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "f0bd58c3-cb9c-46df-9e0a-af1c8d80990e", "content-length": "723", "date": "Wed, 30 Apr 2025 06:44:10 GMT"} })
100242025-04-30T06:44:11.231ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0/component/sp/caboose?firmware_slot=0
100252025-04-30T06:44:11.232ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
100262025-04-30T06:44:11.240ZDEBGtest_utilization (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory result = Ok(())
100272025-04-30T06:44:11.240ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
100282025-04-30T06:44:11.240ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
100292025-04-30T06:44:11.241ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
100302025-04-30T06:44:11.245ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
100312025-04-30T06:44:11.245ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
100322025-04-30T06:44:11.246ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
100332025-04-30T06:44:11.247ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "b732fdf2-a8c2-4624-85bc-b1b39dbbfa2b", "content-length": "109", "date": "Wed, 30 Apr 2025 06:44:10 GMT"} })
100342025-04-30T06:44:11.247ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0/component/sp/caboose?firmware_slot=1
100352025-04-30T06:44:11.251ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "281824ea-0351-4775-8900-d6d2eb3ae925", "content-length": "109", "date": "Wed, 30 Apr 2025 06:44:10 GMT"} })
100362025-04-30T06:44:11.252ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0/component/rot/caboose?firmware_slot=0
100372025-04-30T06:44:11.254ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "5803c532-e223-4099-afb5-0fc4be74f283", "content-length": "169", "date": "Wed, 30 Apr 2025 06:44:10 GMT"} })
100382025-04-30T06:44:11.254ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0/component/rot/caboose?firmware_slot=1
100392025-04-30T06:44:11.256ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "0a8b34c1-c51d-4db8-9468-58ff0ba13484", "content-length": "169", "date": "Wed, 30 Apr 2025 06:44:10 GMT"} })
100402025-04-30T06:44:11.256ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0/component/stage0/caboose?firmware_slot=0
100412025-04-30T06:44:11.258ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "a044fd80-4b36-4c57-bb46-968d072f77dd", "content-length": "178", "date": "Wed, 30 Apr 2025 06:44:10 GMT"} })
100422025-04-30T06:44:11.258ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0/component/stage0/caboose?firmware_slot=1
100432025-04-30T06:44:11.261ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "e66a3e4b-9e4a-442e-9c7c-014ea040b9a3", "content-length": "178", "date": "Wed, 30 Apr 2025 06:44:10 GMT"} })
100442025-04-30T06:44:11.261ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0/component/rot/cmpa
100452025-04-30T06:44:11.262ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "002767af-cadb-407f-8e24-e3bd92e0720d", "content-length": "702", "date": "Wed, 30 Apr 2025 06:44:10 GMT"} })
100462025-04-30T06:44:11.262ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0/component/rot/cfpa
100472025-04-30T06:44:11.264ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "fa6aaac1-da66-4b30-a442-b7b48bb04b0e", "content-length": "727", "date": "Wed, 30 Apr 2025 06:44:10 GMT"} })
100482025-04-30T06:44:11.264ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0/component/rot/cfpa
100492025-04-30T06:44:11.273ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "48248830-2312-4b79-a24c-8b28a71dfdcb", "content-length": "729", "date": "Wed, 30 Apr 2025 06:44:10 GMT"} })
100502025-04-30T06:44:11.273ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0/component/rot/cfpa
100512025-04-30T06:44:11.276ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "8a19773f-4cf1-4fc5-bdb8-d7bc74641458", "content-length": "728", "date": "Wed, 30 Apr 2025 06:44:10 GMT"} })
100522025-04-30T06:44:11.276ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1
100532025-04-30T06:44:11.282ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1", status: 200, headers: {"content-type": "application/json", "x-request-id": "16a06bc3-a2b8-4138-b41d-2e7e4f137cd8", "content-length": "723", "date": "Wed, 30 Apr 2025 06:44:10 GMT"} })
100542025-04-30T06:44:11.282ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1/component/sp/caboose?firmware_slot=0
100552025-04-30T06:44:11.286ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "3b5d1ad5-73ac-408c-833e-a5f2ad75f776", "content-length": "109", "date": "Wed, 30 Apr 2025 06:44:10 GMT"} })
100562025-04-30T06:44:11.286ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1/component/sp/caboose?firmware_slot=1
100572025-04-30T06:44:11.291ZDEBGtest_utilization: attempt to fetch latest inventory collection result = Ok("found")
100582025-04-30T06:44:11.293ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "0d93a2f4-ac91-4fb3-a04e-973d7f7ef541", "content-length": "109", "date": "Wed, 30 Apr 2025 06:44:10 GMT"} })
100592025-04-30T06:44:11.293ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1/component/rot/caboose?firmware_slot=0
100602025-04-30T06:44:11.299ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "4d11fd83-9efb-4a62-a035-700c8cb520d1", "content-length": "169", "date": "Wed, 30 Apr 2025 06:44:10 GMT"} })
100612025-04-30T06:44:11.299ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1/component/rot/caboose?firmware_slot=1
100622025-04-30T06:44:11.312ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "e6dae8cc-a521-4a83-8a67-41c83ad279d9", "content-length": "169", "date": "Wed, 30 Apr 2025 06:44:10 GMT"} })
100632025-04-30T06:44:11.312ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1/component/stage0/caboose?firmware_slot=0
100642025-04-30T06:44:11.316ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "80b3e181-2613-46a1-a4cf-33db884963c0", "content-length": "178", "date": "Wed, 30 Apr 2025 06:44:10 GMT"} })
100652025-04-30T06:44:11.316ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1/component/stage0/caboose?firmware_slot=1
100662025-04-30T06:44:11.318ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "5e78b6ec-f2d1-4365-8807-bb82608572e0", "content-length": "178", "date": "Wed, 30 Apr 2025 06:44:10 GMT"} })
100672025-04-30T06:44:11.319ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1/component/rot/cmpa
100682025-04-30T06:44:11.320ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "49385300-878d-4de6-a0a4-c952cde3aa35", "content-length": "702", "date": "Wed, 30 Apr 2025 06:44:10 GMT"} })
100692025-04-30T06:44:11.320ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1/component/rot/cfpa
100702025-04-30T06:44:11.323ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "77d81e98-d6ae-4b80-975d-acad8f387d03", "content-length": "727", "date": "Wed, 30 Apr 2025 06:44:10 GMT"} })
100712025-04-30T06:44:11.323ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1/component/rot/cfpa
100722025-04-30T06:44:11.326ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "72c0fcfb-a31c-433d-995a-a9cc39f5ca05", "content-length": "729", "date": "Wed, 30 Apr 2025 06:44:10 GMT"} })
100732025-04-30T06:44:11.326ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1/component/rot/cfpa
100742025-04-30T06:44:11.327ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "c1f10dae-4fe8-44a6-9adf-f148e8c59b8b", "content-length": "728", "date": "Wed, 30 Apr 2025 06:44:10 GMT"} })
100752025-04-30T06:44:11.327ZTRACtest_utilization (BackgroundTasks): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet
100762025-04-30T06:44:11.328ZTRACtest_utilization (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
100772025-04-30T06:44:11.328ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
100782025-04-30T06:44:11.328ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
100792025-04-30T06:44:11.329ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
100802025-04-30T06:44:11.333ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
100812025-04-30T06:44:11.339ZDEBGtest_utilization (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet result = Ok(())
100822025-04-30T06:44:11.339ZTRACtest_utilization (BackgroundTasks): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet
100832025-04-30T06:44:11.340ZTRACtest_utilization (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
100842025-04-30T06:44:11.340ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
100852025-04-30T06:44:11.340ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
100862025-04-30T06:44:11.340ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
100872025-04-30T06:44:11.350ZTRACtest_utilization (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory
100882025-04-30T06:44:11.351ZTRACtest_utilization (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
100892025-04-30T06:44:11.351ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
100902025-04-30T06:44:11.351ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
100912025-04-30T06:44:11.353ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
100922025-04-30T06:44:11.355ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
100932025-04-30T06:44:11.360ZDEBGtest_utilization (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet result = Ok(())
100942025-04-30T06:44:11.360ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
100952025-04-30T06:44:11.360ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
100962025-04-30T06:44:11.361ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
100972025-04-30T06:44:11.362ZDEBGtest_utilization (results-sink): inserting 1 samples into database collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
100982025-04-30T06:44:11.363ZDEBGtest_utilization (results-sink-cluster): inserting 1 samples into database collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
100992025-04-30T06:44:11.363ZDEBGtest_utilization (clickhouse-client): unrolling 1 total samples collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 4d1485f0-110e-4161-9a33-c4610d68be5d
101002025-04-30T06:44:11.366ZTRACtest_utilization (clickhouse-client): inserting data collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 4d1485f0-110e-4161-9a33-c4610d68be5d n_columns = 4 n_rows = 2 sql = INSERT INTO oximeter.fields_string FORMAT Native
101012025-04-30T06:44:11.367ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
101022025-04-30T06:44:11.380ZDEBGtest_utilization (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory result = Ok(())
101032025-04-30T06:44:11.382ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
101042025-04-30T06:44:11.383ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
101052025-04-30T06:44:11.384ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
101062025-04-30T06:44:11.439ZDEBGtest_utilization (BackgroundTasks): begin collection from Sled Agent background_task = inventory_collection sled_agent_url = http://[::1]:39365
101072025-04-30T06:44:11.439ZDEBGtest_utilization (BackgroundTasks): client request SledAgent = http://[::1]:39365 background_task = inventory_collection body = None method = GET uri = http://[::1]:39365/inventory
101082025-04-30T06:44:11.441ZINFOtest_utilization (dropshot): accepted connection local_addr = [::1]:39365 remote_addr = [::1]:45403 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
101092025-04-30T06:44:11.441ZTRACtest_utilization (dropshot): incoming request local_addr = [::1]:39365 method = GET remote_addr = [::1]:45403 req_id = fbda6f59-c8a0-408a-b08c-023a36571e74 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /inventory
101102025-04-30T06:44:11.442ZINFOtest_utilization (dropshot): request completed latency_us = 658 local_addr = [::1]:39365 method = GET remote_addr = [::1]:45403 req_id = fbda6f59-c8a0-408a-b08c-023a36571e74 response_code = 200 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /inventory
101112025-04-30T06:44:11.442ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
101122025-04-30T06:44:11.442ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
101132025-04-30T06:44:11.444ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
101142025-04-30T06:44:11.444ZDEBGtest_utilization (BackgroundTasks): client response SledAgent = http://[::1]:39365 background_task = inventory_collection result = Ok(Response { url: "http://[::1]:39365/inventory", status: 200, headers: {"content-type": "application/json", "x-request-id": "fbda6f59-c8a0-408a-b08c-023a36571e74", "content-length": "3141", "date": "Wed, 30 Apr 2025 06:44:10 GMT"} })
101152025-04-30T06:44:11.446ZDEBGtest_utilization (BackgroundTasks): begin collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
101162025-04-30T06:44:11.446ZDEBGtest_utilization (BackgroundTasks): end collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
101172025-04-30T06:44:11.446ZDEBGtest_utilization (BackgroundTasks): finished collection background_task = inventory_collection
101182025-04-30T06:44:11.447ZTRACtest_utilization (BackgroundTasks): authorize begin action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory
101192025-04-30T06:44:11.447ZTRACtest_utilization (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
101202025-04-30T06:44:11.447ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
101212025-04-30T06:44:11.448ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
101222025-04-30T06:44:11.448ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
101232025-04-30T06:44:11.449ZDEBGtest_utilization (clickhouse-client): inserted rows into table collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 4d1485f0-110e-4161-9a33-c4610d68be5d n_rows = 2 table_name = fields_string
101242025-04-30T06:44:11.449ZTRACtest_utilization (clickhouse-client): inserting data collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 4d1485f0-110e-4161-9a33-c4610d68be5d n_columns = 4 n_rows = 1 sql = INSERT INTO oximeter.measurements_i64 FORMAT Native
101252025-04-30T06:44:11.454ZDEBGtest_utilization (clickhouse-client): inserted rows into table collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 4d1485f0-110e-4161-9a33-c4610d68be5d n_rows = 1 table_name = measurements_i64
101262025-04-30T06:44:11.454ZTRACtest_utilization (results-sink): successfully inserted samples collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
101272025-04-30T06:44:11.454ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
101282025-04-30T06:44:11.458ZDEBGtest_utilization (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory result = Ok(())
101292025-04-30T06:44:11.459ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
101302025-04-30T06:44:11.459ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
101312025-04-30T06:44:11.460ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
101322025-04-30T06:44:11.466ZWARNtest_utilization (results-sink-cluster): failed to insert some results into metric DB collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 error = Failed to check out connection to database: No backends found for this service file = oximeter/collector/src/results_sink.rs:92
101332025-04-30T06:44:11.500ZWARNtest_utilization (datastore): Non-retryable transaction failure err = DatabaseError(UniqueViolation, "duplicate key value violates unique constraint \\"inv_caboose_pkey\\"") transaction = inventory_insert_collection
101342025-04-30T06:44:11.501ZWARNtest_utilization (BackgroundTasks): inventory collection failed background_task = inventory_collection error = failed to collect inventory: saving inventory to database: Internal Error: unexpected database error: duplicate key value violates unique constraint "inv_caboose_pkey"
101352025-04-30T06:44:11.501ZDEBGtest_utilization (BackgroundTasks): activation complete background_task = inventory_collection elapsed = 449.858213ms iteration = 22 status = {"error":"failed to collect inventory: saving inventory to database: Internal Error: unexpected database error: duplicate key value violates unique constraint \\"inv_caboose_pkey\\""}
101362025-04-30T06:44:11.501ZDEBGtest_utilization (BackgroundTasks): activating background_task = inventory_collection iteration = 23 reason = Signaled
101372025-04-30T06:44:11.501ZTRACtest_utilization (BackgroundTasks): authorize begin action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory
101382025-04-30T06:44:11.501ZTRACtest_utilization (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
101392025-04-30T06:44:11.501ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
101402025-04-30T06:44:11.501ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
101412025-04-30T06:44:11.502ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
101422025-04-30T06:44:11.502ZDEBGtest_utilization: attempt to fetch latest inventory collection result = Ok("found")
101432025-04-30T06:44:11.505ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
101442025-04-30T06:44:11.519ZDEBGtest_utilization (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory result = Ok(())
101452025-04-30T06:44:11.519ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
101462025-04-30T06:44:11.519ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
101472025-04-30T06:44:11.520ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
101482025-04-30T06:44:11.524ZDEBGtest_utilization (BackgroundTasks): inventory_prune_one: nothing eligible for removal (too few) background_task = inventory_collection candidates = [(13d8b8d9-6c6b-47a6-9190-952544edb75b, 0)]
101492025-04-30T06:44:11.524ZTRACtest_utilization (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal
101502025-04-30T06:44:11.527ZTRACtest_utilization (dns): buffer buffer = Length: 67 (0x43) bytes\n0000: 18 66 01 00 00 01 00 00 00 00 00 01 04 5f 6d 67 .f..........._mg\n0010: 73 04 5f 74 63 70 0d 63 6f 6e 74 72 6f 6c 2d 70 s._tcp.control-p\n0020: 6c 61 6e 65 05 6f 78 69 64 65 08 69 6e 74 65 72 lane.oxide.inter\n0030: 6e 61 6c 00 00 21 00 01 00 00 29 04 d0 00 00 00 nal..!....).....\n0040: 00 00 00 ... kind = dns peer_addr = [::1]:58025 req_id = 04120b4f-de3a-4b8e-839d-ccbfc4500ef7
101512025-04-30T06:44:11.527ZDEBGtest_utilization (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 6246,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_mgs._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_mgs._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 4,\n 95,\n 109,\n 103,\n 115,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:58025 req_id = 04120b4f-de3a-4b8e-839d-ccbfc4500ef7
101522025-04-30T06:44:11.527ZDEBGtest_utilization (store): query key key = _mgs._tcp
101532025-04-30T06:44:11.528ZDEBGtest_utilization (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
101542025-04-30T06:44:11.528ZDEBGtest_utilization (dns): dns response additional_records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal"), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] kind = dns peer_addr = [::1]:58025 query = LowerQuery { name: LowerName(Name("_mgs._tcp.control-plane.oxide.internal.")), original: Query { name: Name("_mgs._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN } } records = [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 48759, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal") })) }] req_id = 04120b4f-de3a-4b8e-839d-ccbfc4500ef7
101552025-04-30T06:44:11.528ZDEBGtest_utilization (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgs._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 48759, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }], valid_until: Instant { tv_sec: 4316, tv_nsec: 327581042 } })
101562025-04-30T06:44:11.528ZTRACtest_utilization (DnsResolver): lookup_service_targets: looking up SRV target
101572025-04-30T06:44:11.529ZTRACtest_utilization (dns): buffer buffer = Length: 108 (0x6c) bytes\n0000: 96 ce 01 00 00 01 00 00 00 00 00 01 2d 64 65 6e ............-den\n0010: 64 72 69 74 65 2d 62 36 64 36 35 33 34 31 2d 31 drite-b6d65341-1\n0020: 36 37 63 2d 34 31 64 66 2d 39 62 35 63 2d 34 31 67c-41df-9b5c-41\n0030: 63 64 65 64 39 39 63 32 32 39 04 68 6f 73 74 0d cded99c229.host.\n0040: 63 6f 6e 74 72 6f 6c 2d 70 6c 61 6e 65 05 6f 78 control-plane.ox\n0050: 69 64 65 08 69 6e 74 65 72 6e 61 6c 00 00 1c 00 ide.internal....\n0060: 01 00 00 29 04 d0 00 00 00 00 00 00 ...)........ kind = dns peer_addr = [::1]:60024 req_id = a48dccde-49e8-45e3-8a49-62e246751150
101582025-04-30T06:44:11.529ZDEBGtest_utilization (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 38606,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n query_type: AAAA,\n query_class: IN,\n },\n },\n original: [\n 45,\n 100,\n 101,\n 110,\n 100,\n 114,\n 105,\n 116,\n 101,\n 45,\n 98,\n 54,\n 100,\n 54,\n 53,\n 51,\n 52,\n 49,\n 45,\n 49,\n 54,\n 55,\n 99,\n 45,\n 52,\n 49,\n 100,\n 102,\n 45,\n 57,\n 98,\n 53,\n 99,\n 45,\n 52,\n 49,\n 99,\n 100,\n 101,\n 100,\n 57,\n 57,\n 99,\n 50,\n 50,\n 57,\n 4,\n 104,\n 111,\n 115,\n 116,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 28,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:60024 req_id = a48dccde-49e8-45e3-8a49-62e246751150
101592025-04-30T06:44:11.529ZDEBGtest_utilization (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
101602025-04-30T06:44:11.529ZDEBGtest_utilization (dns): dns response additional_records = [] kind = dns peer_addr = [::1]:60024 query = LowerQuery { name: LowerName(Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.")), original: Query { name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), query_type: AAAA, query_class: IN } } records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] req_id = a48dccde-49e8-45e3-8a49-62e246751150
101612025-04-30T06:44:11.609ZTRACtest_utilization (DnsResolver): lookup_all_socket_v6 srv dns_name = _clickhouse-admin-keeper._tcp.control-plane.oxide.internal
101622025-04-30T06:44:11.609ZTRACtest_utilization (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory
101632025-04-30T06:44:11.609ZTRACtest_utilization (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
101642025-04-30T06:44:11.609ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
101652025-04-30T06:44:11.609ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
101662025-04-30T06:44:11.610ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
101672025-04-30T06:44:11.611ZTRACtest_utilization (dns): buffer buffer = Length: 87 (0x57) bytes\n0000: 75 18 01 00 00 01 00 00 00 00 00 01 18 5f 63 6c u............_cl\n0010: 69 63 6b 68 6f 75 73 65 2d 61 64 6d 69 6e 2d 6b ickhouse-admin-k\n0020: 65 65 70 65 72 04 5f 74 63 70 0d 63 6f 6e 74 72 eeper._tcp.contr\n0030: 6f 6c 2d 70 6c 61 6e 65 05 6f 78 69 64 65 08 69 ol-plane.oxide.i\n0040: 6e 74 65 72 6e 61 6c 00 00 21 00 01 00 00 29 04 nternal..!....).\n0050: d0 00 00 00 00 00 00 ....... kind = dns peer_addr = [::1]:50586 req_id = 06dd065c-92d3-4425-95da-889bd1f7e63c
101682025-04-30T06:44:11.611ZDEBGtest_utilization (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 29976,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 24,\n 95,\n 99,\n 108,\n 105,\n 99,\n 107,\n 104,\n 111,\n 117,\n 115,\n 101,\n 45,\n 97,\n 100,\n 109,\n 105,\n 110,\n 45,\n 107,\n 101,\n 101,\n 112,\n 101,\n 114,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:50586 req_id = 06dd065c-92d3-4425-95da-889bd1f7e63c
101692025-04-30T06:44:11.612ZDEBGtest_utilization (store): query key key = _clickhouse-admin-keeper._tcp
101702025-04-30T06:44:11.612ZERROtest_utilization (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 29976, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 1, }, query: WireQuery { query: LowerQuery { name: LowerName( Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), ), original: Query { name: Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN, }, }, original: [ 24, 95, 99, 108, 105, 99, 107, 104, 111, 117, 115, 101, 45, 97, 100, 109, 105, 110, 45, 107, 101, 101, 112, 101, 114, 4, 95, 116, 99, 112, 13, 99, 111, 110, 116, 114, 111, 108, 45, 112, 108, 97, 110, 101, 5, 111, 120, 105, 100, 101, 8, 105, 110, 116, 101, 114, 110, 97, 108, 0, 0, 33, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: Some( Edns { rcode_high: 0, version: 0, dnssec_ok: false, max_payload: 1232, options: OPT { options: {}, }, }, ), } NXDOMAIN: no records found for name: "_clickhouse-admin-keeper._tcp.control-plane.oxide.internal." kind = dns peer_addr = [::1]:50586 req_id = 06dd065c-92d3-4425-95da-889bd1f7e63c
101712025-04-30T06:44:11.614ZDEBGtest_utilization (BackgroundTasks): begin collection background_task = inventory_collection
101722025-04-30T06:44:11.614ZDEBGtest_utilization (BackgroundTasks): begin collection from MGS background_task = inventory_collection mgs_url = http://[::1]:48759
101732025-04-30T06:44:11.614ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/ignition
101742025-04-30T06:44:11.614ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
101752025-04-30T06:44:11.625ZDEBGtest_utilization (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory result = Ok(())
101762025-04-30T06:44:11.625ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
101772025-04-30T06:44:11.625ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
101782025-04-30T06:44:11.626ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
101792025-04-30T06:44:11.639ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
101802025-04-30T06:44:11.639ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
101812025-04-30T06:44:11.640ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
101822025-04-30T06:44:11.667ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/ignition", status: 200, headers: {"content-type": "application/json", "x-request-id": "617bbf47-51bd-482a-9540-dea838a2ccce", "content-length": "835", "date": "Wed, 30 Apr 2025 06:44:10 GMT"} })
101832025-04-30T06:44:11.667ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0
101842025-04-30T06:44:11.701ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "929ce25e-f3bb-4e01-898e-4b3d669d51dc", "content-length": "734", "date": "Wed, 30 Apr 2025 06:44:11 GMT"} })
101852025-04-30T06:44:11.702ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0/component/sp/caboose?firmware_slot=0
101862025-04-30T06:44:11.722ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "9f23892e-5f02-4306-9499-24723a041fce", "content-length": "111", "date": "Wed, 30 Apr 2025 06:44:11 GMT"} })
101872025-04-30T06:44:11.724ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0/component/sp/caboose?firmware_slot=1
101882025-04-30T06:44:11.730ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "452faea3-6030-4107-8529-9dfb68a5dbd6", "content-length": "111", "date": "Wed, 30 Apr 2025 06:44:11 GMT"} })
101892025-04-30T06:44:11.730ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0/component/rot/caboose?firmware_slot=0
101902025-04-30T06:44:11.734ZDEBGtest_utilization: attempt to fetch latest inventory collection result = Ok("found")
101912025-04-30T06:44:11.746ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "e19a6233-3756-4ae4-b73e-bb5e2fbfe4c0", "content-length": "170", "date": "Wed, 30 Apr 2025 06:44:11 GMT"} })
101922025-04-30T06:44:11.746ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0/component/rot/caboose?firmware_slot=1
101932025-04-30T06:44:11.752ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "6ca1f34e-dfad-4425-90e0-c5c730cf30e9", "content-length": "170", "date": "Wed, 30 Apr 2025 06:44:11 GMT"} })
101942025-04-30T06:44:11.754ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0/component/stage0/caboose?firmware_slot=0
101952025-04-30T06:44:11.757ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "feae2716-0605-4399-b1a7-0636076daa8c", "content-length": "179", "date": "Wed, 30 Apr 2025 06:44:11 GMT"} })
101962025-04-30T06:44:11.757ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0/component/stage0/caboose?firmware_slot=1
101972025-04-30T06:44:11.771ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "7bdb1d6f-08e5-4f6c-b4df-9d18441c9b0a", "content-length": "179", "date": "Wed, 30 Apr 2025 06:44:11 GMT"} })
101982025-04-30T06:44:11.771ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0/component/rot/cmpa
101992025-04-30T06:44:11.774ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "030bb0ea-0283-4f35-9fbb-804e82b11488", "content-length": "702", "date": "Wed, 30 Apr 2025 06:44:11 GMT"} })
102002025-04-30T06:44:11.774ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0/component/rot/cfpa
102012025-04-30T06:44:11.776ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "cd50ad06-a5c1-400c-9f42-1d3c71b779d8", "content-length": "727", "date": "Wed, 30 Apr 2025 06:44:11 GMT"} })
102022025-04-30T06:44:11.776ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0/component/rot/cfpa
102032025-04-30T06:44:11.777ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "990cf052-4866-4512-9b06-ab8905a624df", "content-length": "729", "date": "Wed, 30 Apr 2025 06:44:11 GMT"} })
102042025-04-30T06:44:11.777ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0/component/rot/cfpa
102052025-04-30T06:44:11.778ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "bf1e0970-f204-4b98-a326-b3146f153b43", "content-length": "728", "date": "Wed, 30 Apr 2025 06:44:11 GMT"} })
102062025-04-30T06:44:11.778ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1
102072025-04-30T06:44:11.779ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1", status: 200, headers: {"content-type": "application/json", "x-request-id": "050178db-5952-4f91-be60-53a861b67040", "content-length": "734", "date": "Wed, 30 Apr 2025 06:44:11 GMT"} })
102082025-04-30T06:44:11.779ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1/component/sp/caboose?firmware_slot=0
102092025-04-30T06:44:11.782ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "04871ee7-4382-4fcc-b754-6e131990dd94", "content-length": "111", "date": "Wed, 30 Apr 2025 06:44:11 GMT"} })
102102025-04-30T06:44:11.782ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1/component/sp/caboose?firmware_slot=1
102112025-04-30T06:44:11.785ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "bf6872a5-a68d-4111-b751-0fac8898ee30", "content-length": "111", "date": "Wed, 30 Apr 2025 06:44:11 GMT"} })
102122025-04-30T06:44:11.785ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1/component/rot/caboose?firmware_slot=0
102132025-04-30T06:44:11.785ZTRACtest_utilization (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory
102142025-04-30T06:44:11.785ZTRACtest_utilization (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
102152025-04-30T06:44:11.785ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
102162025-04-30T06:44:11.785ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
102172025-04-30T06:44:11.787ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
102182025-04-30T06:44:11.790ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
102192025-04-30T06:44:11.801ZDEBGtest_utilization (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory result = Ok(())
102202025-04-30T06:44:11.801ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
102212025-04-30T06:44:11.801ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
102222025-04-30T06:44:11.803ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
102232025-04-30T06:44:11.806ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "0420b34a-e446-4c2b-9041-e045b99beae1", "content-length": "170", "date": "Wed, 30 Apr 2025 06:44:11 GMT"} })
102242025-04-30T06:44:11.807ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1/component/rot/caboose?firmware_slot=1
102252025-04-30T06:44:11.808ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
102262025-04-30T06:44:11.809ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
102272025-04-30T06:44:11.809ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
102282025-04-30T06:44:11.813ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "2bd163ed-37d6-4fe1-b87a-fab1413223ef", "content-length": "170", "date": "Wed, 30 Apr 2025 06:44:11 GMT"} })
102292025-04-30T06:44:11.813ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1/component/stage0/caboose?firmware_slot=0
102302025-04-30T06:44:11.819ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "498a85a8-aa83-47f0-a700-4a9ac3877d8f", "content-length": "179", "date": "Wed, 30 Apr 2025 06:44:11 GMT"} })
102312025-04-30T06:44:11.819ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1/component/stage0/caboose?firmware_slot=1
102322025-04-30T06:44:11.838ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "3609e82a-7ac0-4203-ab18-3d55e3da883f", "content-length": "179", "date": "Wed, 30 Apr 2025 06:44:11 GMT"} })
102332025-04-30T06:44:11.838ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1/component/rot/cmpa
102342025-04-30T06:44:11.840ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "9107d342-516f-46c9-b004-ddac8a23484c", "content-length": "702", "date": "Wed, 30 Apr 2025 06:44:11 GMT"} })
102352025-04-30T06:44:11.840ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1/component/rot/cfpa
102362025-04-30T06:44:11.841ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "9095aea8-953b-405a-bb04-d35f53b826c5", "content-length": "727", "date": "Wed, 30 Apr 2025 06:44:11 GMT"} })
102372025-04-30T06:44:11.842ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1/component/rot/cfpa
102382025-04-30T06:44:11.843ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "19590054-8c18-4fcd-b4bc-e28943d063df", "content-length": "729", "date": "Wed, 30 Apr 2025 06:44:11 GMT"} })
102392025-04-30T06:44:11.843ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1/component/rot/cfpa
102402025-04-30T06:44:11.845ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "53fa1ccd-86b6-445d-b03b-fbdd9f55a80f", "content-length": "728", "date": "Wed, 30 Apr 2025 06:44:11 GMT"} })
102412025-04-30T06:44:11.845ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0
102422025-04-30T06:44:11.847ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "ffe8d46d-3dc5-4089-bd34-7f9c96c1795c", "content-length": "723", "date": "Wed, 30 Apr 2025 06:44:11 GMT"} })
102432025-04-30T06:44:11.847ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0/component/sp/caboose?firmware_slot=0
102442025-04-30T06:44:11.852ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "67d15bf0-b625-4f0a-a0c5-e57bdcf9758e", "content-length": "109", "date": "Wed, 30 Apr 2025 06:44:11 GMT"} })
102452025-04-30T06:44:11.852ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0/component/sp/caboose?firmware_slot=1
102462025-04-30T06:44:11.857ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "4b7d2006-addd-444c-bcfe-01d4cf757688", "content-length": "109", "date": "Wed, 30 Apr 2025 06:44:11 GMT"} })
102472025-04-30T06:44:11.857ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0/component/rot/caboose?firmware_slot=0
102482025-04-30T06:44:11.863ZDEBGtest_utilization: attempt to fetch latest inventory collection result = Ok("found")
102492025-04-30T06:44:11.865ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "eaf5c53d-7038-4897-a29a-843c9ad396c8", "content-length": "169", "date": "Wed, 30 Apr 2025 06:44:11 GMT"} })
102502025-04-30T06:44:11.865ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0/component/rot/caboose?firmware_slot=1
102512025-04-30T06:44:11.871ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "dcc606f5-a525-4ac4-abf3-05f8d6bea9a5", "content-length": "169", "date": "Wed, 30 Apr 2025 06:44:11 GMT"} })
102522025-04-30T06:44:11.871ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0/component/stage0/caboose?firmware_slot=0
102532025-04-30T06:44:11.877ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "0bf82778-4c7b-4728-9ae6-cf76f6dbdbea", "content-length": "178", "date": "Wed, 30 Apr 2025 06:44:11 GMT"} })
102542025-04-30T06:44:11.877ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0/component/stage0/caboose?firmware_slot=1
102552025-04-30T06:44:11.884ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "3c41e951-af84-4516-86b6-571458acdb29", "content-length": "178", "date": "Wed, 30 Apr 2025 06:44:11 GMT"} })
102562025-04-30T06:44:11.885ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0/component/rot/cmpa
102572025-04-30T06:44:11.888ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "ed5f55b2-6328-4d23-8425-b7da0266e382", "content-length": "702", "date": "Wed, 30 Apr 2025 06:44:11 GMT"} })
102582025-04-30T06:44:11.889ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0/component/rot/cfpa
102592025-04-30T06:44:11.892ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "a72474ca-edca-4346-9610-aadfd55a9e77", "content-length": "727", "date": "Wed, 30 Apr 2025 06:44:11 GMT"} })
102602025-04-30T06:44:11.892ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0/component/rot/cfpa
102612025-04-30T06:44:11.897ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "8f59e2c2-7002-4eab-8fe8-4f12f867f7dd", "content-length": "729", "date": "Wed, 30 Apr 2025 06:44:11 GMT"} })
102622025-04-30T06:44:11.897ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0/component/rot/cfpa
102632025-04-30T06:44:11.903ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "c6efd06a-18fb-4bc4-86ac-a684481cff11", "content-length": "728", "date": "Wed, 30 Apr 2025 06:44:11 GMT"} })
102642025-04-30T06:44:11.903ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1
102652025-04-30T06:44:11.906ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1", status: 200, headers: {"content-type": "application/json", "x-request-id": "3a60839b-bc9a-4c4f-810b-cfcbe802a5cc", "content-length": "723", "date": "Wed, 30 Apr 2025 06:44:11 GMT"} })
102662025-04-30T06:44:11.907ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1/component/sp/caboose?firmware_slot=0
102672025-04-30T06:44:11.909ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "7ef7ff6b-1b79-429d-b0e5-de17ee22ec54", "content-length": "109", "date": "Wed, 30 Apr 2025 06:44:11 GMT"} })
102682025-04-30T06:44:11.909ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1/component/sp/caboose?firmware_slot=1
102692025-04-30T06:44:11.910ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "2d7f9341-6c11-4c44-905f-e733062828d0", "content-length": "109", "date": "Wed, 30 Apr 2025 06:44:11 GMT"} })
102702025-04-30T06:44:11.910ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1/component/rot/caboose?firmware_slot=0
102712025-04-30T06:44:11.912ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "5809f52c-8cb1-4942-9053-a427d7829149", "content-length": "169", "date": "Wed, 30 Apr 2025 06:44:11 GMT"} })
102722025-04-30T06:44:11.913ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1/component/rot/caboose?firmware_slot=1
102732025-04-30T06:44:11.914ZTRACtest_utilization (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory
102742025-04-30T06:44:11.914ZTRACtest_utilization (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
102752025-04-30T06:44:11.915ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
102762025-04-30T06:44:11.915ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
102772025-04-30T06:44:11.916ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
102782025-04-30T06:44:11.919ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "0430e379-9236-4802-b654-f48664d3854e", "content-length": "169", "date": "Wed, 30 Apr 2025 06:44:11 GMT"} })
102792025-04-30T06:44:11.920ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1/component/stage0/caboose?firmware_slot=0
102802025-04-30T06:44:11.920ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
102812025-04-30T06:44:11.930ZDEBGtest_utilization (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory result = Ok(())
102822025-04-30T06:44:11.930ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
102832025-04-30T06:44:11.930ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
102842025-04-30T06:44:11.931ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
102852025-04-30T06:44:11.935ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
102862025-04-30T06:44:11.936ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
102872025-04-30T06:44:11.936ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
102882025-04-30T06:44:11.938ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "0994dce2-f40c-4bbb-ac7c-55e85e34c120", "content-length": "178", "date": "Wed, 30 Apr 2025 06:44:11 GMT"} })
102892025-04-30T06:44:11.938ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1/component/stage0/caboose?firmware_slot=1
102902025-04-30T06:44:11.943ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "df655c01-1f22-4404-becd-2e8f013c6bf6", "content-length": "178", "date": "Wed, 30 Apr 2025 06:44:11 GMT"} })
102912025-04-30T06:44:11.943ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1/component/rot/cmpa
102922025-04-30T06:44:11.944ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "49ac2bbb-5a3f-41b2-86d5-f8cab802e41c", "content-length": "702", "date": "Wed, 30 Apr 2025 06:44:11 GMT"} })
102932025-04-30T06:44:11.944ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1/component/rot/cfpa
102942025-04-30T06:44:11.947ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "3241ba18-47cd-4fff-92b4-b856e4eadb04", "content-length": "727", "date": "Wed, 30 Apr 2025 06:44:11 GMT"} })
102952025-04-30T06:44:11.947ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1/component/rot/cfpa
102962025-04-30T06:44:11.948ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "2a4f78af-ddc5-4046-8a9f-6e6a4a9f4036", "content-length": "729", "date": "Wed, 30 Apr 2025 06:44:11 GMT"} })
102972025-04-30T06:44:11.948ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1/component/rot/cfpa
102982025-04-30T06:44:11.949ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "45bb1866-e998-4aa3-8be1-9363d9ea29bb", "content-length": "728", "date": "Wed, 30 Apr 2025 06:44:11 GMT"} })
102992025-04-30T06:44:11.949ZTRACtest_utilization (BackgroundTasks): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet
103002025-04-30T06:44:11.949ZTRACtest_utilization (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
103012025-04-30T06:44:11.949ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
103022025-04-30T06:44:11.949ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
103032025-04-30T06:44:11.950ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
103042025-04-30T06:44:11.955ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
103052025-04-30T06:44:11.963ZDEBGtest_utilization (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet result = Ok(())
103062025-04-30T06:44:11.963ZTRACtest_utilization (BackgroundTasks): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet
103072025-04-30T06:44:11.963ZTRACtest_utilization (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
103082025-04-30T06:44:11.963ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
103092025-04-30T06:44:11.963ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
103102025-04-30T06:44:11.963ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
103112025-04-30T06:44:11.964ZINFOtest_utilization (oximeter-agent): refreshing list of producers from Nexus collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 file = oximeter/collector/src/agent.rs:562
103122025-04-30T06:44:11.965ZDEBGtest_utilization (oximeter): client request body = None method = GET uri = http://127.0.0.1:32956/metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?sort_by=id_ascending
103132025-04-30T06:44:11.966ZTRACtest_utilization (dropshot_internal): incoming request local_addr = 127.0.0.1:32956 method = GET remote_addr = 127.0.0.1:55797 req_id = ad7cea0b-c847-416f-89fd-d428c58cf98a uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?sort_by=id_ascending
103142025-04-30T06:44:11.967ZTRACtest_utilization (dropshot_internal): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = 127.0.0.1:32956 method = GET remote_addr = 127.0.0.1:55797 req_id = ad7cea0b-c847-416f-89fd-d428c58cf98a resource = Database uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?sort_by=id_ascending
103152025-04-30T06:44:11.967ZDEBGtest_utilization (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = 127.0.0.1:32956 method = GET remote_addr = 127.0.0.1:55797 req_id = ad7cea0b-c847-416f-89fd-d428c58cf98a roles = RoleSet { roles: {} } uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?sort_by=id_ascending
103162025-04-30T06:44:11.967ZDEBGtest_utilization (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = 127.0.0.1:32956 method = GET remote_addr = 127.0.0.1:55797 req_id = ad7cea0b-c847-416f-89fd-d428c58cf98a resource = Database result = Ok(()) uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?sort_by=id_ascending
103172025-04-30T06:44:11.968ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
103182025-04-30T06:44:11.973ZDEBGtest_utilization (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet result = Ok(())
103192025-04-30T06:44:11.973ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
103202025-04-30T06:44:11.974ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
103212025-04-30T06:44:11.974ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
103222025-04-30T06:44:11.981ZINFOtest_utilization (dropshot_internal): request completed latency_us = 11641 local_addr = 127.0.0.1:32956 method = GET remote_addr = 127.0.0.1:55797 req_id = ad7cea0b-c847-416f-89fd-d428c58cf98a response_code = 200 uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?sort_by=id_ascending
103232025-04-30T06:44:11.981ZDEBGtest_utilization (oximeter): client response result = Ok(Response { url: "http://127.0.0.1:32956/metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?sort_by=id_ascending", status: 200, headers: {"content-type": "application/json", "x-request-id": "ad7cea0b-c847-416f-89fd-d428c58cf98a", "content-length": "405", "date": "Wed, 30 Apr 2025 06:44:11 GMT"} })
103242025-04-30T06:44:11.982ZDEBGtest_utilization (oximeter): client request body = None method = GET uri = http://127.0.0.1:32956/metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJhYzU1ZDhkOS1hMjk4LTQ1MzQtODk4Yi1iZWVlMWM5ZTRlNTIifX0%3D
103252025-04-30T06:44:11.982ZTRACtest_utilization (dropshot_internal): incoming request local_addr = 127.0.0.1:32956 method = GET remote_addr = 127.0.0.1:55797 req_id = 2e445e61-f765-42ed-930c-c3906331bcb8 uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJhYzU1ZDhkOS1hMjk4LTQ1MzQtODk4Yi1iZWVlMWM5ZTRlNTIifX0%3D
103262025-04-30T06:44:11.982ZTRACtest_utilization (dropshot_internal): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = 127.0.0.1:32956 method = GET remote_addr = 127.0.0.1:55797 req_id = 2e445e61-f765-42ed-930c-c3906331bcb8 resource = Database uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJhYzU1ZDhkOS1hMjk4LTQ1MzQtODk4Yi1iZWVlMWM5ZTRlNTIifX0%3D
103272025-04-30T06:44:11.982ZDEBGtest_utilization (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = 127.0.0.1:32956 method = GET remote_addr = 127.0.0.1:55797 req_id = 2e445e61-f765-42ed-930c-c3906331bcb8 roles = RoleSet { roles: {} } uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJhYzU1ZDhkOS1hMjk4LTQ1MzQtODk4Yi1iZWVlMWM5ZTRlNTIifX0%3D
103282025-04-30T06:44:11.982ZDEBGtest_utilization (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = 127.0.0.1:32956 method = GET remote_addr = 127.0.0.1:55797 req_id = 2e445e61-f765-42ed-930c-c3906331bcb8 resource = Database result = Ok(()) uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJhYzU1ZDhkOS1hMjk4LTQ1MzQtODk4Yi1iZWVlMWM5ZTRlNTIifX0%3D
103292025-04-30T06:44:11.984ZINFOtest_utilization (dropshot_internal): request completed latency_us = 5511 local_addr = 127.0.0.1:32956 method = GET remote_addr = 127.0.0.1:55797 req_id = 2e445e61-f765-42ed-930c-c3906331bcb8 response_code = 200 uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJhYzU1ZDhkOS1hMjk4LTQ1MzQtODk4Yi1iZWVlMWM5ZTRlNTIifX0%3D
103302025-04-30T06:44:11.985ZDEBGtest_utilization (oximeter): client response result = Ok(Response { url: "http://127.0.0.1:32956/metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJhYzU1ZDhkOS1hMjk4LTQ1MzQtODk4Yi1iZWVlMWM5ZTRlNTIifX0%3D", status: 200, headers: {"content-type": "application/json", "x-request-id": "2e445e61-f765-42ed-930c-c3906331bcb8", "content-length": "29", "date": "Wed, 30 Apr 2025 06:44:11 GMT"} })
103312025-04-30T06:44:11.985ZTRACtest_utilization (oximeter-agent): ignoring request to update existing metric producer, since the endpoint information is the same as the existing collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
103322025-04-30T06:44:11.985ZTRACtest_utilization (oximeter-agent): ignoring request to update existing metric producer, since the endpoint information is the same as the existing collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = ac55d8d9-a298-4534-898b-beee1c9e4e52
103332025-04-30T06:44:11.985ZINFOtest_utilization (oximeter-agent): refreshed list of producers from Nexus collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 file = oximeter/collector/src/agent.rs:637 n_current_tasks = 2 n_pruned_tasks = 0
103342025-04-30T06:44:11.998ZDEBGtest_utilization (collection-task): sent timer-based collection request to the collection task collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
103352025-04-30T06:44:11.998ZDEBGtest_utilization (collection-task): collecting from producer collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
103362025-04-30T06:44:12.003ZDEBGtest_utilization: attempt to fetch latest inventory collection result = Ok("found")
103372025-04-30T06:44:12.004ZTRACtest_utilization (collection-task): sent collection request to producer collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
103382025-04-30T06:44:12.005ZDEBGtest_utilization (collection-task): collected results from producer collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 n_results = 1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
103392025-04-30T06:44:12.005ZTRACtest_utilization (collection-task): forwarded results to main collection loop collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
103402025-04-30T06:44:12.005ZTRACtest_utilization (collection-task): top of inner collection loop, waiting for next request collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
103412025-04-30T06:44:12.055ZTRACtest_utilization (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory
103422025-04-30T06:44:12.055ZTRACtest_utilization (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
103432025-04-30T06:44:12.055ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
103442025-04-30T06:44:12.056ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
103452025-04-30T06:44:12.065ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
103462025-04-30T06:44:12.119ZDEBGtest_utilization (BackgroundTasks): begin collection from Sled Agent background_task = inventory_collection sled_agent_url = http://[::1]:39365
103472025-04-30T06:44:12.119ZDEBGtest_utilization (BackgroundTasks): client request SledAgent = http://[::1]:39365 background_task = inventory_collection body = None method = GET uri = http://[::1]:39365/inventory
103482025-04-30T06:44:12.120ZINFOtest_utilization (dropshot): accepted connection local_addr = [::1]:39365 remote_addr = [::1]:50379 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
103492025-04-30T06:44:12.121ZTRACtest_utilization (dropshot): incoming request local_addr = [::1]:39365 method = GET remote_addr = [::1]:50379 req_id = 5dd2e390-6a46-4557-895d-24682c8baa5e sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /inventory
103502025-04-30T06:44:12.122ZINFOtest_utilization (dropshot): request completed latency_us = 908 local_addr = [::1]:39365 method = GET remote_addr = [::1]:50379 req_id = 5dd2e390-6a46-4557-895d-24682c8baa5e response_code = 200 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /inventory
103512025-04-30T06:44:12.122ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
103522025-04-30T06:44:12.131ZDEBGtest_utilization (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory result = Ok(())
103532025-04-30T06:44:12.131ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
103542025-04-30T06:44:12.131ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
103552025-04-30T06:44:12.131ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
103562025-04-30T06:44:12.132ZDEBGtest_utilization (BackgroundTasks): client response SledAgent = http://[::1]:39365 background_task = inventory_collection result = Ok(Response { url: "http://[::1]:39365/inventory", status: 200, headers: {"content-type": "application/json", "x-request-id": "5dd2e390-6a46-4557-895d-24682c8baa5e", "content-length": "3141", "date": "Wed, 30 Apr 2025 06:44:11 GMT"} })
103572025-04-30T06:44:12.137ZDEBGtest_utilization (BackgroundTasks): begin collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
103582025-04-30T06:44:12.137ZDEBGtest_utilization (BackgroundTasks): end collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
103592025-04-30T06:44:12.137ZDEBGtest_utilization (BackgroundTasks): finished collection background_task = inventory_collection
103602025-04-30T06:44:12.142ZTRACtest_utilization (BackgroundTasks): authorize begin action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory
103612025-04-30T06:44:12.142ZTRACtest_utilization (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
103622025-04-30T06:44:12.142ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
103632025-04-30T06:44:12.142ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
103642025-04-30T06:44:12.143ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
103652025-04-30T06:44:12.147ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
103662025-04-30T06:44:12.147ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
103672025-04-30T06:44:12.148ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
103682025-04-30T06:44:12.148ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
103692025-04-30T06:44:12.154ZDEBGtest_utilization (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory result = Ok(())
103702025-04-30T06:44:12.154ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
103712025-04-30T06:44:12.155ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
103722025-04-30T06:44:12.157ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
103732025-04-30T06:44:12.213ZWARNtest_utilization (datastore): Non-retryable transaction failure err = DatabaseError(UniqueViolation, "duplicate key value violates unique constraint \\"inv_caboose_pkey\\"") transaction = inventory_insert_collection
103742025-04-30T06:44:12.214ZWARNtest_utilization (BackgroundTasks): inventory collection failed background_task = inventory_collection error = failed to collect inventory: saving inventory to database: Internal Error: unexpected database error: duplicate key value violates unique constraint "inv_caboose_pkey"
103752025-04-30T06:44:12.214ZDEBGtest_utilization (BackgroundTasks): activation complete background_task = inventory_collection elapsed = 712.448157ms iteration = 23 status = {"error":"failed to collect inventory: saving inventory to database: Internal Error: unexpected database error: duplicate key value violates unique constraint \\"inv_caboose_pkey\\""}
103762025-04-30T06:44:12.214ZDEBGtest_utilization (BackgroundTasks): activating background_task = inventory_collection iteration = 24 reason = Signaled
103772025-04-30T06:44:12.214ZTRACtest_utilization (BackgroundTasks): authorize begin action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory
103782025-04-30T06:44:12.214ZTRACtest_utilization (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
103792025-04-30T06:44:12.214ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
103802025-04-30T06:44:12.214ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
103812025-04-30T06:44:12.216ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
103822025-04-30T06:44:12.229ZDEBGtest_utilization (BackgroundTasks): activating background_task = nat_v4_garbage_collector iteration = 2 reason = Timeout
103832025-04-30T06:44:12.229ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database
103842025-04-30T06:44:12.229ZDEBGtest_utilization (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
103852025-04-30T06:44:12.230ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
103862025-04-30T06:44:12.235ZDEBGtest_utilization (BackgroundTasks): activating background_task = bfd_manager iteration = 2 reason = Timeout
103872025-04-30T06:44:12.235ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database
103882025-04-30T06:44:12.235ZDEBGtest_utilization (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
103892025-04-30T06:44:12.235ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
103902025-04-30T06:44:12.235ZDEBGtest_utilization (BackgroundTasks): activating background_task = phantom_disks iteration = 2 reason = Timeout
103912025-04-30T06:44:12.235ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
103922025-04-30T06:44:12.239ZDEBGtest_utilization (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory result = Ok(())
103932025-04-30T06:44:12.239ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
103942025-04-30T06:44:12.239ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
103952025-04-30T06:44:12.240ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
103962025-04-30T06:44:12.243ZDEBGtest_utilization (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 3 reason = Timeout
103972025-04-30T06:44:12.244ZDEBGtest_utilization (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 916.496\u{b5}s iteration = 3 status = {"error":"task disabled"}
103982025-04-30T06:44:12.244ZDEBGtest_utilization (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 2 reason = Timeout
103992025-04-30T06:44:12.244ZTRACtest_utilization (BackgroundTasks): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Fleet
104002025-04-30T06:44:12.244ZTRACtest_utilization (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = switch_port_config_manager resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
104012025-04-30T06:44:12.244ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database
104022025-04-30T06:44:12.244ZDEBGtest_utilization (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
104032025-04-30T06:44:12.245ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
104042025-04-30T06:44:12.245ZDEBGtest_utilization (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 2 reason = Timeout
104052025-04-30T06:44:12.245ZTRACtest_utilization (BackgroundTasks): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory
104062025-04-30T06:44:12.245ZTRACtest_utilization (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = service_zone_nat_tracker resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
104072025-04-30T06:44:12.245ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database
104082025-04-30T06:44:12.245ZDEBGtest_utilization (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
104092025-04-30T06:44:12.246ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
104102025-04-30T06:44:12.246ZDEBGtest_utilization (BackgroundTasks): activating background_task = v2p_manager iteration = 3 reason = Timeout
104112025-04-30T06:44:12.246ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database
104122025-04-30T06:44:12.246ZDEBGtest_utilization (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
104132025-04-30T06:44:12.247ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
104142025-04-30T06:44:12.247ZDEBGtest_utilization (BackgroundTasks): activating background_task = instance_watcher iteration = 2 reason = Timeout
104152025-04-30T06:44:12.247ZTRACtest_utilization (BackgroundTasks): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Fleet
104162025-04-30T06:44:12.247ZTRACtest_utilization (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = instance_watcher resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
104172025-04-30T06:44:12.247ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database
104182025-04-30T06:44:12.247ZDEBGtest_utilization (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
104192025-04-30T06:44:12.248ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
104202025-04-30T06:44:12.248ZDEBGtest_utilization (BackgroundTasks): activating background_task = instance_updater iteration = 2 reason = Timeout
104212025-04-30T06:44:12.248ZINFOtest_utilization (BackgroundTasks): background instance updater explicitly disabled background_task = instance_updater
104222025-04-30T06:44:12.248ZDEBGtest_utilization (BackgroundTasks): activation complete background_task = instance_updater elapsed = 301.561\u{b5}s iteration = 2 status = {"destroyed_active_vmms":0,"disabled":true,"failed_active_vmms":0,"query_errors":[],"saga_errors":[],"sagas_completed":0,"sagas_started":0,"terminated_active_migrations":0}
104232025-04-30T06:44:12.248ZTRACtest_utilization (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal
104242025-04-30T06:44:12.249ZDEBGtest_utilization: attempt to fetch latest inventory collection result = Ok("found")
104252025-04-30T06:44:12.249ZDEBGtest_utilization (BackgroundTasks): activating background_task = vpc_route_manager iteration = 3 reason = Timeout
104262025-04-30T06:44:12.250ZINFOtest_utilization (BackgroundTasks): VPC route manager running background_task = vpc_route_manager
104272025-04-30T06:44:12.250ZTRACtest_utilization (BackgroundTasks): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet
104282025-04-30T06:44:12.250ZTRACtest_utilization (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = vpc_route_manager resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
104292025-04-30T06:44:12.250ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database
104302025-04-30T06:44:12.250ZDEBGtest_utilization (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
104312025-04-30T06:44:12.250ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
104322025-04-30T06:44:12.251ZTRACtest_utilization (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal
104332025-04-30T06:44:12.253ZTRACtest_utilization (dns): buffer buffer = Length: 72 (0x48) bytes\n0000: 42 2b 01 00 00 01 00 00 00 00 00 01 09 5f 64 65 B+..........._de\n0010: 6e 64 72 69 74 65 04 5f 74 63 70 0d 63 6f 6e 74 ndrite._tcp.cont\n0020: 72 6f 6c 2d 70 6c 61 6e 65 05 6f 78 69 64 65 08 rol-plane.oxide.\n0030: 69 6e 74 65 72 6e 61 6c 00 00 21 00 01 00 00 29 internal..!....)\n0040: 04 d0 00 00 00 00 00 00 ........ kind = dns peer_addr = [::1]:63119 req_id = 6b17dc71-22d7-4668-89a4-877f5bfc27e0
104342025-04-30T06:44:12.253ZDEBGtest_utilization (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 16939,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_dendrite._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_dendrite._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 9,\n 95,\n 100,\n 101,\n 110,\n 100,\n 114,\n 105,\n 116,\n 101,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:63119 req_id = 6b17dc71-22d7-4668-89a4-877f5bfc27e0
104352025-04-30T06:44:12.254ZDEBGtest_utilization (store): query key key = _dendrite._tcp
104362025-04-30T06:44:12.254ZDEBGtest_utilization (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
104372025-04-30T06:44:12.254ZDEBGtest_utilization (dns): dns response additional_records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal"), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] kind = dns peer_addr = [::1]:63119 query = LowerQuery { name: LowerName(Name("_dendrite._tcp.control-plane.oxide.internal.")), original: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN } } records = [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 47974, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal") })) }] req_id = 6b17dc71-22d7-4668-89a4-877f5bfc27e0
104382025-04-30T06:44:12.255ZDEBGtest_utilization (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 21.795065ms iteration = 2 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
104392025-04-30T06:44:12.255ZDEBGtest_utilization (BackgroundTasks): inventory_prune_one: nothing eligible for removal (too few) background_task = inventory_collection candidates = [(13d8b8d9-6c6b-47a6-9190-952544edb75b, 0)]
104402025-04-30T06:44:12.255ZTRACtest_utilization (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal
104412025-04-30T06:44:12.256ZDEBGtest_utilization (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
104422025-04-30T06:44:12.259ZDEBGtest_utilization (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
104432025-04-30T06:44:12.259ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database
104442025-04-30T06:44:12.259ZDEBGtest_utilization (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
104452025-04-30T06:44:12.261ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
104462025-04-30T06:44:12.262ZDEBGtest_utilization (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
104472025-04-30T06:44:12.267ZDEBGtest_utilization (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
104482025-04-30T06:44:12.271ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database
104492025-04-30T06:44:12.271ZDEBGtest_utilization (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
104502025-04-30T06:44:12.273ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
104512025-04-30T06:44:12.274ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database
104522025-04-30T06:44:12.274ZDEBGtest_utilization (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
104532025-04-30T06:44:12.275ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
104542025-04-30T06:44:12.278ZDEBGtest_utilization (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
104552025-04-30T06:44:12.287ZDEBGtest_utilization (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Fleet result = Ok(())
104562025-04-30T06:44:12.287ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database
104572025-04-30T06:44:12.287ZDEBGtest_utilization (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
104582025-04-30T06:44:12.288ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
104592025-04-30T06:44:12.289ZDEBGtest_utilization (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
104602025-04-30T06:44:12.299ZDEBGtest_utilization (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
104612025-04-30T06:44:12.304ZTRACtest_utilization (BackgroundTasks): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet
104622025-04-30T06:44:12.304ZTRACtest_utilization (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = vpc_route_manager resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
104632025-04-30T06:44:12.304ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database
104642025-04-30T06:44:12.304ZDEBGtest_utilization (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
104652025-04-30T06:44:12.305ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
104662025-04-30T06:44:12.305ZTRACtest_utilization (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory
104672025-04-30T06:44:12.305ZTRACtest_utilization (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
104682025-04-30T06:44:12.305ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
104692025-04-30T06:44:12.305ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
104702025-04-30T06:44:12.310ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
104712025-04-30T06:44:12.315ZDEBGtest_utilization (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 47974, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }], valid_until: Instant { tv_sec: 4317, tv_nsec: 113142052 } })
104722025-04-30T06:44:12.315ZTRACtest_utilization (DnsResolver): lookup_service_targets: looking up SRV target
104732025-04-30T06:44:12.315ZTRACtest_utilization (dns): buffer buffer = Length: 72 (0x48) bytes\n0000: 7e 76 01 00 00 01 00 00 00 00 00 01 09 5f 64 65 ~v..........._de\n0010: 6e 64 72 69 74 65 04 5f 74 63 70 0d 63 6f 6e 74 ndrite._tcp.cont\n0020: 72 6f 6c 2d 70 6c 61 6e 65 05 6f 78 69 64 65 08 rol-plane.oxide.\n0030: 69 6e 74 65 72 6e 61 6c 00 00 21 00 01 00 00 29 internal..!....)\n0040: 04 d0 00 00 00 00 00 00 ........ kind = dns peer_addr = [::1]:54660 req_id = 760311b7-f865-4312-b357-4f24dc2dc9ef
104742025-04-30T06:44:12.315ZDEBGtest_utilization (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 32374,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_dendrite._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_dendrite._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 9,\n 95,\n 100,\n 101,\n 110,\n 100,\n 114,\n 105,\n 116,\n 101,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:54660 req_id = 760311b7-f865-4312-b357-4f24dc2dc9ef
104752025-04-30T06:44:12.316ZDEBGtest_utilization (store): query key key = _dendrite._tcp
104762025-04-30T06:44:12.316ZDEBGtest_utilization (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
104772025-04-30T06:44:12.316ZDEBGtest_utilization (dns): dns response additional_records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal"), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] kind = dns peer_addr = [::1]:54660 query = LowerQuery { name: LowerName(Name("_dendrite._tcp.control-plane.oxide.internal.")), original: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN } } records = [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 47974, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal") })) }] req_id = 760311b7-f865-4312-b357-4f24dc2dc9ef
104782025-04-30T06:44:12.317ZTRACtest_utilization (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal
104792025-04-30T06:44:12.318ZDEBGtest_utilization (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 47974, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }], valid_until: Instant { tv_sec: 4317, tv_nsec: 117468291 } })
104802025-04-30T06:44:12.318ZTRACtest_utilization (DnsResolver): lookup_service_targets: looking up SRV target
104812025-04-30T06:44:12.318ZTRACtest_utilization (dns): buffer buffer = Length: 67 (0x43) bytes\n0000: 9f 21 01 00 00 01 00 00 00 00 00 01 04 5f 6d 67 .!..........._mg\n0010: 73 04 5f 74 63 70 0d 63 6f 6e 74 72 6f 6c 2d 70 s._tcp.control-p\n0020: 6c 61 6e 65 05 6f 78 69 64 65 08 69 6e 74 65 72 lane.oxide.inter\n0030: 6e 61 6c 00 00 21 00 01 00 00 29 04 d0 00 00 00 nal..!....).....\n0040: 00 00 00 ... kind = dns peer_addr = [::1]:63210 req_id = 4419e2c6-401a-463e-8ca2-086b34df8639
104822025-04-30T06:44:12.318ZDEBGtest_utilization (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 40737,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_mgs._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_mgs._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 4,\n 95,\n 109,\n 103,\n 115,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:63210 req_id = 4419e2c6-401a-463e-8ca2-086b34df8639
104832025-04-30T06:44:12.319ZDEBGtest_utilization (store): query key key = _mgs._tcp
104842025-04-30T06:44:12.319ZDEBGtest_utilization (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
104852025-04-30T06:44:12.319ZDEBGtest_utilization (dns): dns response additional_records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal"), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] kind = dns peer_addr = [::1]:63210 query = LowerQuery { name: LowerName(Name("_mgs._tcp.control-plane.oxide.internal.")), original: Query { name: Name("_mgs._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN } } records = [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 48759, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal") })) }] req_id = 4419e2c6-401a-463e-8ca2-086b34df8639
104862025-04-30T06:44:12.320ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database
104872025-04-30T06:44:12.320ZDEBGtest_utilization (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
104882025-04-30T06:44:12.321ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
104892025-04-30T06:44:12.323ZDEBGtest_utilization (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgs._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 48759, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }], valid_until: Instant { tv_sec: 4317, tv_nsec: 122630497 } })
104902025-04-30T06:44:12.323ZTRACtest_utilization (DnsResolver): lookup_service_targets: looking up SRV target
104912025-04-30T06:44:12.323ZTRACtest_utilization (dns): buffer buffer = Length: 108 (0x6c) bytes\n0000: 4d 49 01 00 00 01 00 00 00 00 00 01 2d 64 65 6e MI..........-den\n0010: 64 72 69 74 65 2d 62 36 64 36 35 33 34 31 2d 31 drite-b6d65341-1\n0020: 36 37 63 2d 34 31 64 66 2d 39 62 35 63 2d 34 31 67c-41df-9b5c-41\n0030: 63 64 65 64 39 39 63 32 32 39 04 68 6f 73 74 0d cded99c229.host.\n0040: 63 6f 6e 74 72 6f 6c 2d 70 6c 61 6e 65 05 6f 78 control-plane.ox\n0050: 69 64 65 08 69 6e 74 65 72 6e 61 6c 00 00 1c 00 ide.internal....\n0060: 01 00 00 29 04 d0 00 00 00 00 00 00 ...)........ kind = dns peer_addr = [::1]:57602 req_id = 3ab960b0-1338-4bf4-aca7-ff3925c3a933
104922025-04-30T06:44:12.323ZDEBGtest_utilization (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 19785,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n query_type: AAAA,\n query_class: IN,\n },\n },\n original: [\n 45,\n 100,\n 101,\n 110,\n 100,\n 114,\n 105,\n 116,\n 101,\n 45,\n 98,\n 54,\n 100,\n 54,\n 53,\n 51,\n 52,\n 49,\n 45,\n 49,\n 54,\n 55,\n 99,\n 45,\n 52,\n 49,\n 100,\n 102,\n 45,\n 57,\n 98,\n 53,\n 99,\n 45,\n 52,\n 49,\n 99,\n 100,\n 101,\n 100,\n 57,\n 57,\n 99,\n 50,\n 50,\n 57,\n 4,\n 104,\n 111,\n 115,\n 116,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 28,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:57602 req_id = 3ab960b0-1338-4bf4-aca7-ff3925c3a933
104932025-04-30T06:44:12.324ZDEBGtest_utilization (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
104942025-04-30T06:44:12.324ZDEBGtest_utilization (dns): dns response additional_records = [] kind = dns peer_addr = [::1]:57602 query = LowerQuery { name: LowerName(Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.")), original: Query { name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), query_type: AAAA, query_class: IN } } records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] req_id = 3ab960b0-1338-4bf4-aca7-ff3925c3a933
104952025-04-30T06:44:12.324ZTRACtest_utilization (dns): buffer buffer = Length: 72 (0x48) bytes\n0000: c7 2d 01 00 00 01 00 00 00 00 00 01 09 5f 64 65 .-..........._de\n0010: 6e 64 72 69 74 65 04 5f 74 63 70 0d 63 6f 6e 74 ndrite._tcp.cont\n0020: 72 6f 6c 2d 70 6c 61 6e 65 05 6f 78 69 64 65 08 rol-plane.oxide.\n0030: 69 6e 74 65 72 6e 61 6c 00 00 21 00 01 00 00 29 internal..!....)\n0040: 04 d0 00 00 00 00 00 00 ........ kind = dns peer_addr = [::1]:56441 req_id = d7f586c5-1522-4861-9985-e61296287ca6
104962025-04-30T06:44:12.324ZDEBGtest_utilization (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 50989,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_dendrite._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_dendrite._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 9,\n 95,\n 100,\n 101,\n 110,\n 100,\n 114,\n 105,\n 116,\n 101,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:56441 req_id = d7f586c5-1522-4861-9985-e61296287ca6
104972025-04-30T06:44:12.324ZDEBGtest_utilization (store): query key key = _dendrite._tcp
104982025-04-30T06:44:12.324ZDEBGtest_utilization (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
104992025-04-30T06:44:12.324ZDEBGtest_utilization (dns): dns response additional_records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal"), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] kind = dns peer_addr = [::1]:56441 query = LowerQuery { name: LowerName(Name("_dendrite._tcp.control-plane.oxide.internal.")), original: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN } } records = [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 47974, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal") })) }] req_id = d7f586c5-1522-4861-9985-e61296287ca6
105002025-04-30T06:44:12.325ZDEBGtest_utilization (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
105012025-04-30T06:44:12.328ZDEBGtest_utilization (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
105022025-04-30T06:44:12.328ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database
105032025-04-30T06:44:12.328ZDEBGtest_utilization (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
105042025-04-30T06:44:12.328ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
105052025-04-30T06:44:12.328ZTRACtest_utilization (BackgroundTasks): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet
105062025-04-30T06:44:12.328ZTRACtest_utilization (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = v2p_manager resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
105072025-04-30T06:44:12.328ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database
105082025-04-30T06:44:12.328ZDEBGtest_utilization (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
105092025-04-30T06:44:12.329ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
105102025-04-30T06:44:12.329ZINFOtest_utilization (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_v4_garbage_collector
105112025-04-30T06:44:12.379ZINFOtest_utilization (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_v4_garbage_collector zone_address = ::1
105122025-04-30T06:44:12.379ZDEBGtest_utilization (MgsClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[::1]:12225/local/switch-id
105132025-04-30T06:44:12.380ZDEBGtest_utilization (MgsClient): client response background_task = nat_v4_garbage_collector result = Err(reqwest::Error { kind: Request, url: "http://[::1]:12225/local/switch-id", source: hyper_util::client::legacy::Error(Connect, ConnectError("tcp connect error", Os { code: 146, kind: ConnectionRefused, message: "Connection refused" })) })
105142025-04-30T06:44:12.380ZWARNtest_utilization (BackgroundTasks): failed to identify switch slot for dendrite background_task = nat_v4_garbage_collector reason = Communication Error: error sending request for url (http://[::1]:12225/local/switch-id): client error (Connect): tcp connect error: Connection refused (os error 146): Connection refused (os error 146) zone_address = ::1
105152025-04-30T06:44:12.381ZWARNtest_utilization (BackgroundTasks): Failed to map switch zone addr: Communication Error: error sending request for url (http://[::1]:12225/local/switch-id), retrying background_task = nat_v4_garbage_collector
105162025-04-30T06:44:12.381ZDEBGtest_utilization (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 47974, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }], valid_until: Instant { tv_sec: 4317, tv_nsec: 180510679 } })
105172025-04-30T06:44:12.381ZTRACtest_utilization (DnsResolver): lookup_service_targets: looking up SRV target
105182025-04-30T06:44:12.471ZTRACtest_utilization (dns): buffer buffer = Length: 108 (0x6c) bytes\n0000: f6 17 01 00 00 01 00 00 00 00 00 01 2d 64 65 6e ............-den\n0010: 64 72 69 74 65 2d 62 36 64 36 35 33 34 31 2d 31 drite-b6d65341-1\n0020: 36 37 63 2d 34 31 64 66 2d 39 62 35 63 2d 34 31 67c-41df-9b5c-41\n0030: 63 64 65 64 39 39 63 32 32 39 04 68 6f 73 74 0d cded99c229.host.\n0040: 63 6f 6e 74 72 6f 6c 2d 70 6c 61 6e 65 05 6f 78 control-plane.ox\n0050: 69 64 65 08 69 6e 74 65 72 6e 61 6c 00 00 1c 00 ide.internal....\n0060: 01 00 00 29 04 d0 00 00 00 00 00 00 ...)........ kind = dns peer_addr = [::1]:60690 req_id = c165a93b-8abf-4eae-ab94-95228f645e1f
105192025-04-30T06:44:12.471ZDEBGtest_utilization (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 62999,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n query_type: AAAA,\n query_class: IN,\n },\n },\n original: [\n 45,\n 100,\n 101,\n 110,\n 100,\n 114,\n 105,\n 116,\n 101,\n 45,\n 98,\n 54,\n 100,\n 54,\n 53,\n 51,\n 52,\n 49,\n 45,\n 49,\n 54,\n 55,\n 99,\n 45,\n 52,\n 49,\n 100,\n 102,\n 45,\n 57,\n 98,\n 53,\n 99,\n 45,\n 52,\n 49,\n 99,\n 100,\n 101,\n 100,\n 57,\n 57,\n 99,\n 50,\n 50,\n 57,\n 4,\n 104,\n 111,\n 115,\n 116,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 28,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:60690 req_id = c165a93b-8abf-4eae-ab94-95228f645e1f
105202025-04-30T06:44:12.472ZDEBGtest_utilization (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
105212025-04-30T06:44:12.472ZDEBGtest_utilization (dns): dns response additional_records = [] kind = dns peer_addr = [::1]:60690 query = LowerQuery { name: LowerName(Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.")), original: Query { name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), query_type: AAAA, query_class: IN } } records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] req_id = c165a93b-8abf-4eae-ab94-95228f645e1f
105222025-04-30T06:44:12.473ZTRACtest_utilization (BackgroundTasks): checking on VMM background_task = instance_watcher instance_id = d7bd2deb-ee27-4661-9e9a-524d734cd6ec propolis_id = 9e8dbed7-faa3-4634-886b-3a005dd8e99b sled_id = b6d65341-167c-41df-9b5c-41cded99c229 vmm_id = 9e8dbed7-faa3-4634-886b-3a005dd8e99b
105232025-04-30T06:44:12.473ZDEBGtest_utilization (BackgroundTasks): client request SledAgent = b6d65341-167c-41df-9b5c-41cded99c229 background_task = instance_watcher body = None method = GET uri = http://[::1]:39365/vmms/9e8dbed7-faa3-4634-886b-3a005dd8e99b/state
105242025-04-30T06:44:12.475ZINFOtest_utilization (dropshot): accepted connection local_addr = [::1]:39365 remote_addr = [::1]:52842 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
105252025-04-30T06:44:12.475ZINFOtest_utilization (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager
105262025-04-30T06:44:12.525ZINFOtest_utilization (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager zone_address = ::1
105272025-04-30T06:44:12.525ZDEBGtest_utilization (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[::1]:12225/local/switch-id
105282025-04-30T06:44:12.525ZDEBGtest_utilization (MgsClient): client response background_task = bfd_manager result = Err(reqwest::Error { kind: Request, url: "http://[::1]:12225/local/switch-id", source: hyper_util::client::legacy::Error(Connect, ConnectError("tcp connect error", Os { code: 146, kind: ConnectionRefused, message: "Connection refused" })) })
105292025-04-30T06:44:12.525ZWARNtest_utilization (BackgroundTasks): failed to identify switch slot for dendrite background_task = bfd_manager reason = Communication Error: error sending request for url (http://[::1]:12225/local/switch-id): client error (Connect): tcp connect error: Connection refused (os error 146): Connection refused (os error 146) zone_address = ::1
105302025-04-30T06:44:12.526ZWARNtest_utilization (BackgroundTasks): Failed to map switch zone addr: Communication Error: error sending request for url (http://[::1]:12225/local/switch-id), retrying background_task = bfd_manager
105312025-04-30T06:44:12.526ZDEBGtest_utilization (results-sink): inserting 1 samples into database collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
105322025-04-30T06:44:12.526ZDEBGtest_utilization (results-sink-cluster): inserting 1 samples into database collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
105332025-04-30T06:44:12.527ZTRACtest_utilization (dns): buffer buffer = Length: 108 (0x6c) bytes\n0000: 97 3c 01 00 00 01 00 00 00 00 00 01 2d 64 65 6e .<..........-den\n0010: 64 72 69 74 65 2d 62 36 64 36 35 33 34 31 2d 31 drite-b6d65341-1\n0020: 36 37 63 2d 34 31 64 66 2d 39 62 35 63 2d 34 31 67c-41df-9b5c-41\n0030: 63 64 65 64 39 39 63 32 32 39 04 68 6f 73 74 0d cded99c229.host.\n0040: 63 6f 6e 74 72 6f 6c 2d 70 6c 61 6e 65 05 6f 78 control-plane.ox\n0050: 69 64 65 08 69 6e 74 65 72 6e 61 6c 00 00 1c 00 ide.internal....\n0060: 01 00 00 29 04 d0 00 00 00 00 00 00 ...)........ kind = dns peer_addr = [::1]:53105 req_id = fd4c106d-c90d-4378-b473-d2540b9ea6f9
105342025-04-30T06:44:12.527ZDEBGtest_utilization (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 38716,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n query_type: AAAA,\n query_class: IN,\n },\n },\n original: [\n 45,\n 100,\n 101,\n 110,\n 100,\n 114,\n 105,\n 116,\n 101,\n 45,\n 98,\n 54,\n 100,\n 54,\n 53,\n 51,\n 52,\n 49,\n 45,\n 49,\n 54,\n 55,\n 99,\n 45,\n 52,\n 49,\n 100,\n 102,\n 45,\n 57,\n 98,\n 53,\n 99,\n 45,\n 52,\n 49,\n 99,\n 100,\n 101,\n 100,\n 57,\n 57,\n 99,\n 50,\n 50,\n 57,\n 4,\n 104,\n 111,\n 115,\n 116,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 28,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:53105 req_id = fd4c106d-c90d-4378-b473-d2540b9ea6f9
105352025-04-30T06:44:12.528ZDEBGtest_utilization (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
105362025-04-30T06:44:12.528ZDEBGtest_utilization (dns): dns response additional_records = [] kind = dns peer_addr = [::1]:53105 query = LowerQuery { name: LowerName(Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.")), original: Query { name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), query_type: AAAA, query_class: IN } } records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] req_id = fd4c106d-c90d-4378-b473-d2540b9ea6f9
105372025-04-30T06:44:12.528ZDEBGtest_utilization (clickhouse-client): unrolling 1 total samples collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 4d1485f0-110e-4161-9a33-c4610d68be5d
105382025-04-30T06:44:12.530ZTRACtest_utilization (clickhouse-client): inserting data collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 4d1485f0-110e-4161-9a33-c4610d68be5d n_columns = 4 n_rows = 2 sql = INSERT INTO oximeter.fields_string FORMAT Native
105392025-04-30T06:44:12.531ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
105402025-04-30T06:44:12.537ZDEBGtest_utilization (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory result = Ok(())
105412025-04-30T06:44:12.537ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
105422025-04-30T06:44:12.537ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
105432025-04-30T06:44:12.537ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
105442025-04-30T06:44:12.614ZTRACtest_utilization (DnsResolver): lookup_all_socket_v6 srv dns_name = _clickhouse-admin-keeper._tcp.control-plane.oxide.internal
105452025-04-30T06:44:12.615ZTRACtest_utilization (dropshot): incoming request local_addr = [::1]:39365 method = GET remote_addr = [::1]:52842 req_id = 5429db16-52fa-4caa-ae15-371fb839cbdd sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /vmms/9e8dbed7-faa3-4634-886b-3a005dd8e99b/state
105462025-04-30T06:44:12.616ZTRACtest_utilization (dns): buffer buffer = Length: 108 (0x6c) bytes\n0000: 32 e3 01 00 00 01 00 00 00 00 00 01 2d 64 65 6e 2...........-den\n0010: 64 72 69 74 65 2d 62 36 64 36 35 33 34 31 2d 31 drite-b6d65341-1\n0020: 36 37 63 2d 34 31 64 66 2d 39 62 35 63 2d 34 31 67c-41df-9b5c-41\n0030: 63 64 65 64 39 39 63 32 32 39 04 68 6f 73 74 0d cded99c229.host.\n0040: 63 6f 6e 74 72 6f 6c 2d 70 6c 61 6e 65 05 6f 78 control-plane.ox\n0050: 69 64 65 08 69 6e 74 65 72 6e 61 6c 00 00 1c 00 ide.internal....\n0060: 01 00 00 29 04 d0 00 00 00 00 00 00 ...)........ kind = dns peer_addr = [::1]:57176 req_id = 096f239c-a43a-450d-9c7c-0ee37b05e362
105472025-04-30T06:44:12.616ZDEBGtest_utilization (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 13027,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n query_type: AAAA,\n query_class: IN,\n },\n },\n original: [\n 45,\n 100,\n 101,\n 110,\n 100,\n 114,\n 105,\n 116,\n 101,\n 45,\n 98,\n 54,\n 100,\n 54,\n 53,\n 51,\n 52,\n 49,\n 45,\n 49,\n 54,\n 55,\n 99,\n 45,\n 52,\n 49,\n 100,\n 102,\n 45,\n 57,\n 98,\n 53,\n 99,\n 45,\n 52,\n 49,\n 99,\n 100,\n 101,\n 100,\n 57,\n 57,\n 99,\n 50,\n 50,\n 57,\n 4,\n 104,\n 111,\n 115,\n 116,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 28,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:57176 req_id = 096f239c-a43a-450d-9c7c-0ee37b05e362
105482025-04-30T06:44:12.616ZDEBGtest_utilization (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
105492025-04-30T06:44:12.616ZDEBGtest_utilization (dns): dns response additional_records = [] kind = dns peer_addr = [::1]:57176 query = LowerQuery { name: LowerName(Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.")), original: Query { name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), query_type: AAAA, query_class: IN } } records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] req_id = 096f239c-a43a-450d-9c7c-0ee37b05e362
105502025-04-30T06:44:12.617ZINFOtest_utilization (dropshot): request completed latency_us = 1440 local_addr = [::1]:39365 method = GET remote_addr = [::1]:52842 req_id = 5429db16-52fa-4caa-ae15-371fb839cbdd response_code = 200 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /vmms/9e8dbed7-faa3-4634-886b-3a005dd8e99b/state
105512025-04-30T06:44:12.691ZINFOtest_utilization (BackgroundTasks): VPC route manager sled b6d65341-167c-41df-9b5c-41cded99c229 background_task = vpc_route_manager
105522025-04-30T06:44:12.691ZDEBGtest_utilization (BackgroundTasks): client request SledAgent = b6d65341-167c-41df-9b5c-41cded99c229 background_task = vpc_route_manager body = None method = GET uri = http://[::1]:39365/vpc-routes
105532025-04-30T06:44:12.692ZDEBGtest_utilization (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
105542025-04-30T06:44:12.699ZDEBGtest_utilization (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
105552025-04-30T06:44:12.699ZTRACtest_utilization (BackgroundTasks): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet
105562025-04-30T06:44:12.699ZTRACtest_utilization (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = v2p_manager resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
105572025-04-30T06:44:12.699ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database
105582025-04-30T06:44:12.699ZDEBGtest_utilization (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
105592025-04-30T06:44:12.701ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
105602025-04-30T06:44:12.702ZINFOtest_utilization (dropshot): accepted connection local_addr = [::1]:39365 remote_addr = [::1]:57012 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
105612025-04-30T06:44:12.703ZDEBGtest_utilization (clickhouse-client): inserted rows into table collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 4d1485f0-110e-4161-9a33-c4610d68be5d n_rows = 2 table_name = fields_string
105622025-04-30T06:44:12.703ZTRACtest_utilization (clickhouse-client): inserting data collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 4d1485f0-110e-4161-9a33-c4610d68be5d n_columns = 4 n_rows = 1 sql = INSERT INTO oximeter.measurements_i64 FORMAT Native
105632025-04-30T06:44:12.705ZINFOtest_utilization (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager rack_id = c19a698f-c6f9-4a17-ae30-20d711b8f7dc
105642025-04-30T06:44:12.783ZINFOtest_utilization (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager rack_id = c19a698f-c6f9-4a17-ae30-20d711b8f7dc zone_address = ::1
105652025-04-30T06:44:12.783ZDEBGtest_utilization (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = c19a698f-c6f9-4a17-ae30-20d711b8f7dc uri = http://[::1]:12225/local/switch-id
105662025-04-30T06:44:12.785ZDEBGtest_utilization (MgsClient): client response background_task = switch_port_config_manager rack_id = c19a698f-c6f9-4a17-ae30-20d711b8f7dc result = Err(reqwest::Error { kind: Request, url: "http://[::1]:12225/local/switch-id", source: hyper_util::client::legacy::Error(Connect, ConnectError("tcp connect error", Os { code: 146, kind: ConnectionRefused, message: "Connection refused" })) })
105672025-04-30T06:44:12.785ZWARNtest_utilization (BackgroundTasks): failed to identify switch slot for dendrite background_task = switch_port_config_manager rack_id = c19a698f-c6f9-4a17-ae30-20d711b8f7dc reason = Communication Error: error sending request for url (http://[::1]:12225/local/switch-id): client error (Connect): tcp connect error: Connection refused (os error 146): Connection refused (os error 146) zone_address = ::1
105682025-04-30T06:44:12.786ZWARNtest_utilization (BackgroundTasks): Failed to map switch zone addr: Communication Error: error sending request for url (http://[::1]:12225/local/switch-id), retrying background_task = switch_port_config_manager rack_id = c19a698f-c6f9-4a17-ae30-20d711b8f7dc
105692025-04-30T06:44:12.788ZDEBGtest_utilization (BackgroundTasks): client response SledAgent = b6d65341-167c-41df-9b5c-41cded99c229 background_task = instance_watcher result = Ok(Response { url: "http://[::1]:39365/vmms/9e8dbed7-faa3-4634-886b-3a005dd8e99b/state", status: 200, headers: {"content-type": "application/json", "x-request-id": "5429db16-52fa-4caa-ae15-371fb839cbdd", "content-length": "131", "date": "Wed, 30 Apr 2025 06:44:11 GMT"} })
105702025-04-30T06:44:12.788ZDEBGtest_utilization (BackgroundTasks): updating instance state background_task = instance_watcher instance_id = d7bd2deb-ee27-4661-9e9a-524d734cd6ec sled_id = b6d65341-167c-41df-9b5c-41cded99c229 state = SledVmmState { vmm_state: VmmRuntimeState { state: Starting, gen: Generation(2), time_updated: 2025-04-30T06:43:59.549025164Z }, migration_in: None, migration_out: None } vmm_id = 9e8dbed7-faa3-4634-886b-3a005dd8e99b
105712025-04-30T06:44:12.788ZINFOtest_utilization (BackgroundTasks): received new VMM runtime state from sled agent background_task = instance_watcher instance_id = d7bd2deb-ee27-4661-9e9a-524d734cd6ec migration_state = Migrations { migration_in: None, migration_out: None } propolis_id = 9e8dbed7-faa3-4634-886b-3a005dd8e99b sled_id = b6d65341-167c-41df-9b5c-41cded99c229 vmm_id = 9e8dbed7-faa3-4634-886b-3a005dd8e99b vmm_state = VmmRuntimeState { state: Starting, gen: Generation(2), time_updated: 2025-04-30T06:43:59.549025164Z }
105722025-04-30T06:44:12.788ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher instance_id = d7bd2deb-ee27-4661-9e9a-524d734cd6ec resource = Database sled_id = b6d65341-167c-41df-9b5c-41cded99c229 vmm_id = 9e8dbed7-faa3-4634-886b-3a005dd8e99b
105732025-04-30T06:44:12.788ZDEBGtest_utilization (BackgroundTasks): roles background_task = instance_watcher instance_id = d7bd2deb-ee27-4661-9e9a-524d734cd6ec roles = RoleSet { roles: {} } sled_id = b6d65341-167c-41df-9b5c-41cded99c229 vmm_id = 9e8dbed7-faa3-4634-886b-3a005dd8e99b
105742025-04-30T06:44:12.790ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher instance_id = d7bd2deb-ee27-4661-9e9a-524d734cd6ec resource = Database result = Ok(()) sled_id = b6d65341-167c-41df-9b5c-41cded99c229 vmm_id = 9e8dbed7-faa3-4634-886b-3a005dd8e99b
105752025-04-30T06:44:12.790ZWARNtest_utilization (results-sink-cluster): failed to insert some results into metric DB collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 error = Failed to check out connection to database: No backends found for this service file = oximeter/collector/src/results_sink.rs:92
105762025-04-30T06:44:12.794ZDEBGtest_utilization (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
105772025-04-30T06:44:12.805ZDEBGtest_utilization (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
105782025-04-30T06:44:12.805ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database
105792025-04-30T06:44:12.806ZDEBGtest_utilization (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
105802025-04-30T06:44:12.806ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
105812025-04-30T06:44:12.811ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
105822025-04-30T06:44:12.811ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
105832025-04-30T06:44:12.813ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
105842025-04-30T06:44:12.817ZTRACtest_utilization (dropshot): incoming request local_addr = [::1]:39365 method = GET remote_addr = [::1]:57012 req_id = b8ec323f-10c4-4ac3-9aa0-cd2860cd9c67 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /vpc-routes
105852025-04-30T06:44:12.817ZTRACtest_utilization (dns): buffer buffer = Length: 87 (0x57) bytes\n0000: de b8 01 00 00 01 00 00 00 00 00 01 18 5f 63 6c ............._cl\n0010: 69 63 6b 68 6f 75 73 65 2d 61 64 6d 69 6e 2d 6b ickhouse-admin-k\n0020: 65 65 70 65 72 04 5f 74 63 70 0d 63 6f 6e 74 72 eeper._tcp.contr\n0030: 6f 6c 2d 70 6c 61 6e 65 05 6f 78 69 64 65 08 69 ol-plane.oxide.i\n0040: 6e 74 65 72 6e 61 6c 00 00 21 00 01 00 00 29 04 nternal..!....).\n0050: d0 00 00 00 00 00 00 ....... kind = dns peer_addr = [::1]:60086 req_id = 1aca483c-b0ae-478a-a66d-c1ac3e0007bb
105862025-04-30T06:44:12.817ZDEBGtest_utilization (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 57016,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 24,\n 95,\n 99,\n 108,\n 105,\n 99,\n 107,\n 104,\n 111,\n 117,\n 115,\n 101,\n 45,\n 97,\n 100,\n 109,\n 105,\n 110,\n 45,\n 107,\n 101,\n 101,\n 112,\n 101,\n 114,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:60086 req_id = 1aca483c-b0ae-478a-a66d-c1ac3e0007bb
105872025-04-30T06:44:12.818ZDEBGtest_utilization (store): query key key = _clickhouse-admin-keeper._tcp
105882025-04-30T06:44:12.818ZERROtest_utilization (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 57016, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 1, }, query: WireQuery { query: LowerQuery { name: LowerName( Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), ), original: Query { name: Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN, }, }, original: [ 24, 95, 99, 108, 105, 99, 107, 104, 111, 117, 115, 101, 45, 97, 100, 109, 105, 110, 45, 107, 101, 101, 112, 101, 114, 4, 95, 116, 99, 112, 13, 99, 111, 110, 116, 114, 111, 108, 45, 112, 108, 97, 110, 101, 5, 111, 120, 105, 100, 101, 8, 105, 110, 116, 101, 114, 110, 97, 108, 0, 0, 33, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: Some( Edns { rcode_high: 0, version: 0, dnssec_ok: false, max_payload: 1232, options: OPT { options: {}, }, }, ), } NXDOMAIN: no records found for name: "_clickhouse-admin-keeper._tcp.control-plane.oxide.internal." kind = dns peer_addr = [::1]:60086 req_id = 1aca483c-b0ae-478a-a66d-c1ac3e0007bb
105892025-04-30T06:44:12.822ZINFOtest_utilization (dropshot): request completed latency_us = 4943 local_addr = [::1]:39365 method = GET remote_addr = [::1]:57012 req_id = b8ec323f-10c4-4ac3-9aa0-cd2860cd9c67 response_code = 200 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /vpc-routes
105902025-04-30T06:44:12.891ZDEBGtest_utilization (BackgroundTasks): client request SledAgent = b6d65341-167c-41df-9b5c-41cded99c229 background_task = v2p_manager body = None method = GET uri = http://[::1]:39365/v2p
105912025-04-30T06:44:12.897ZINFOtest_utilization (dropshot): accepted connection local_addr = [::1]:39365 remote_addr = [::1]:39058 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
105922025-04-30T06:44:12.898ZDEBGtest_utilization (clickhouse-client): inserted rows into table collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 4d1485f0-110e-4161-9a33-c4610d68be5d n_rows = 1 table_name = measurements_i64
105932025-04-30T06:44:12.898ZTRACtest_utilization (results-sink): successfully inserted samples collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
105942025-04-30T06:44:12.898ZDEBGtest_utilization (BackgroundTasks): begin collection background_task = inventory_collection
105952025-04-30T06:44:12.898ZDEBGtest_utilization (BackgroundTasks): begin collection from MGS background_task = inventory_collection mgs_url = http://[::1]:48759
105962025-04-30T06:44:12.898ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/ignition
105972025-04-30T06:44:12.899ZDEBGtest_utilization (BackgroundTasks): client response SledAgent = b6d65341-167c-41df-9b5c-41cded99c229 background_task = vpc_route_manager result = Ok(Response { url: "http://[::1]:39365/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "b8ec323f-10c4-4ac3-9aa0-cd2860cd9c67", "content-length": "214", "date": "Wed, 30 Apr 2025 06:44:12 GMT"} })
105982025-04-30T06:44:12.899ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database
105992025-04-30T06:44:12.899ZDEBGtest_utilization (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
106002025-04-30T06:44:12.900ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
106012025-04-30T06:44:12.902ZTRACtest_utilization (dropshot): incoming request local_addr = [::1]:39365 method = GET remote_addr = [::1]:39058 req_id = 63cda9af-c5d1-4635-a40e-01e99f597ade sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /v2p
106022025-04-30T06:44:12.903ZINFOtest_utilization (dropshot): request completed latency_us = 690 local_addr = [::1]:39365 method = GET remote_addr = [::1]:39058 req_id = 63cda9af-c5d1-4635-a40e-01e99f597ade response_code = 200 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /v2p
106032025-04-30T06:44:12.904ZINFOtest_utilization (BackgroundTasks): all instance checks complete background_task = instance_watcher pruned_instances = 0 total_completed = 1 total_failed = 0 total_incomplete = 0 total_instances = 1 update_sagas_queued = 0
106042025-04-30T06:44:12.907ZDEBGtest_utilization (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 660.602827ms iteration = 2 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{"starting":1},"pruned_instances":0,"total_instances":1,"update_sagas_queued":0}
106052025-04-30T06:44:12.913ZDEBGtest_utilization (BackgroundTasks): client response SledAgent = b6d65341-167c-41df-9b5c-41cded99c229 background_task = v2p_manager result = Ok(Response { url: "http://[::1]:39365/v2p", status: 200, headers: {"content-type": "application/json", "x-request-id": "63cda9af-c5d1-4635-a40e-01e99f597ade", "content-length": "103", "date": "Wed, 30 Apr 2025 06:44:12 GMT"} })
106062025-04-30T06:44:12.913ZINFOtest_utilization (BackgroundTasks): found opte v2p mappings background_task = v2p_manager interfaces = {VirtualNetworkInterfaceHost { physical_host_ip: ::1, virtual_ip: 172.30.0.5, virtual_mac: MacAddr(MacAddr6([168, 64, 37, 240, 0, 0])), vni: Vni(11911937) }} sled = sim-b6d65341
106072025-04-30T06:44:12.913ZINFOtest_utilization (BackgroundTasks): v2p mappings to delete background_task = v2p_manager mappings = [] sled = sim-b6d65341
106082025-04-30T06:44:12.913ZINFOtest_utilization (BackgroundTasks): v2p mappings to add background_task = v2p_manager mappings = [] sled = sim-b6d65341
106092025-04-30T06:44:12.917ZDEBGtest_utilization (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 671.180694ms iteration = 3 status = {}
106102025-04-30T06:44:12.923ZINFOtest_utilization (BackgroundTasks): computed internet gateway mappings for sled assocs = {64c48731-6cc1-46ff-99f2-86f6c481d0c6: {10.0.0.0: {5567c57c-3b46-4ca0-968f-03632bf48df0}}} background_task = vpc_route_manager sled = sim-b6d65341
106112025-04-30T06:44:12.925ZDEBGtest_utilization (BackgroundTasks): client request SledAgent = b6d65341-167c-41df-9b5c-41cded99c229 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[::1]:39365/eip-gateways
106122025-04-30T06:44:12.926ZTRACtest_utilization (dropshot): incoming request local_addr = [::1]:39365 method = PUT remote_addr = [::1]:57012 req_id = 32700565-9e7b-4857-9b07-784c21de4fb7 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /eip-gateways
106132025-04-30T06:44:12.927ZINFOtest_utilization (dropshot): request completed latency_us = 384 local_addr = [::1]:39365 method = PUT remote_addr = [::1]:57012 req_id = 32700565-9e7b-4857-9b07-784c21de4fb7 response_code = 204 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /eip-gateways
106142025-04-30T06:44:12.934ZDEBGtest_utilization (BackgroundTasks): client response SledAgent = b6d65341-167c-41df-9b5c-41cded99c229 background_task = vpc_route_manager result = Ok(Response { url: "http://[::1]:39365/eip-gateways", status: 204, headers: {"x-request-id": "32700565-9e7b-4857-9b07-784c21de4fb7", "date": "Wed, 30 Apr 2025 06:44:12 GMT"} })
106152025-04-30T06:44:12.934ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database
106162025-04-30T06:44:12.934ZDEBGtest_utilization (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
106172025-04-30T06:44:12.936ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
106182025-04-30T06:44:12.940ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/ignition", status: 200, headers: {"content-type": "application/json", "x-request-id": "74bdf673-72e3-49db-bd2c-30d94d9014ca", "content-length": "835", "date": "Wed, 30 Apr 2025 06:44:12 GMT"} })
106192025-04-30T06:44:12.941ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0
106202025-04-30T06:44:12.941ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database
106212025-04-30T06:44:12.941ZDEBGtest_utilization (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
106222025-04-30T06:44:12.942ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
106232025-04-30T06:44:12.951ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database
106242025-04-30T06:44:12.951ZDEBGtest_utilization (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
106252025-04-30T06:44:12.953ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
106262025-04-30T06:44:12.953ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database
106272025-04-30T06:44:12.953ZDEBGtest_utilization (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
106282025-04-30T06:44:12.954ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
106292025-04-30T06:44:12.964ZINFOtest_utilization (BackgroundTasks): VPC route manager sled b6d65341-167c-41df-9b5c-41cded99c229 push not needed background_task = vpc_route_manager
106302025-04-30T06:44:12.964ZDEBGtest_utilization (BackgroundTasks): client request SledAgent = b6d65341-167c-41df-9b5c-41cded99c229 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[::1]:39365/vpc-routes
106312025-04-30T06:44:12.967ZTRACtest_utilization (dropshot): incoming request local_addr = [::1]:39365 method = PUT remote_addr = [::1]:57012 req_id = 36e1f063-83b2-4395-b2e4-030b2fdb69c5 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /vpc-routes
106322025-04-30T06:44:12.968ZINFOtest_utilization (dropshot): request completed latency_us = 988 local_addr = [::1]:39365 method = PUT remote_addr = [::1]:57012 req_id = 36e1f063-83b2-4395-b2e4-030b2fdb69c5 response_code = 204 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /vpc-routes
106332025-04-30T06:44:12.968ZTRACtest_utilization (BackgroundTasks): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: b6d65341-167c-41df-9b5c-41cded99c229, lookup_type: ById(b6d65341-167c-41df-9b5c-41cded99c229) }
106342025-04-30T06:44:12.968ZTRACtest_utilization (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = service_zone_nat_tracker resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
106352025-04-30T06:44:12.969ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database
106362025-04-30T06:44:12.969ZDEBGtest_utilization (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
106372025-04-30T06:44:12.969ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
106382025-04-30T06:44:12.972ZDEBGtest_utilization (BackgroundTasks): client response SledAgent = b6d65341-167c-41df-9b5c-41cded99c229 background_task = vpc_route_manager result = Ok(Response { url: "http://[::1]:39365/vpc-routes", status: 204, headers: {"x-request-id": "36e1f063-83b2-4395-b2e4-030b2fdb69c5", "date": "Wed, 30 Apr 2025 06:44:12 GMT"} })
106392025-04-30T06:44:12.974ZDEBGtest_utilization (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 724.237396ms iteration = 3 status = {}
106402025-04-30T06:44:12.976ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "79073609-5d66-4bd7-a6ba-e5af2bd42f88", "content-length": "734", "date": "Wed, 30 Apr 2025 06:44:12 GMT"} })
106412025-04-30T06:44:12.976ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0/component/sp/caboose?firmware_slot=0
106422025-04-30T06:44:12.976ZDEBGtest_utilization (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
106432025-04-30T06:44:12.983ZDEBGtest_utilization (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: b6d65341-167c-41df-9b5c-41cded99c229, lookup_type: ById(b6d65341-167c-41df-9b5c-41cded99c229) } result = Ok(())
106442025-04-30T06:44:12.983ZERROtest_utilization (BackgroundTasks): ipv6 addresses for service zone nat not implemented background_task = service_zone_nat_tracker
106452025-04-30T06:44:12.983ZERROtest_utilization (BackgroundTasks): generated config for fewer than the minimum allowed number of dns zones background_task = service_zone_nat_tracker
106462025-04-30T06:44:12.983ZDEBGtest_utilization (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 738.011199ms iteration = 2 status = {"error":"generated config for fewer than the minimum allowed number of dns zones"}
106472025-04-30T06:44:12.995ZDEBGtest_utilization (collection-task): sent timer-based collection request to the collection task collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
106482025-04-30T06:44:12.995ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "0c222b48-89ea-4823-9b46-9718796a01f3", "content-length": "111", "date": "Wed, 30 Apr 2025 06:44:12 GMT"} })
106492025-04-30T06:44:12.996ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0/component/sp/caboose?firmware_slot=1
106502025-04-30T06:44:12.996ZDEBGtest_utilization (collection-task): collecting from producer collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
106512025-04-30T06:44:13.001ZTRACtest_utilization (collection-task): sent collection request to producer collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
106522025-04-30T06:44:13.001ZDEBGtest_utilization (collection-task): collected results from producer collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 n_results = 1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
106532025-04-30T06:44:13.001ZTRACtest_utilization (collection-task): forwarded results to main collection loop collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
106542025-04-30T06:44:13.001ZTRACtest_utilization (collection-task): top of inner collection loop, waiting for next request collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
106552025-04-30T06:44:13.002ZDEBGtest_utilization: attempt to fetch latest inventory collection result = Ok("found")
106562025-04-30T06:44:13.004ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "d1e9b558-ce03-4589-9428-2c3636d16afc", "content-length": "111", "date": "Wed, 30 Apr 2025 06:44:12 GMT"} })
106572025-04-30T06:44:13.004ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0/component/rot/caboose?firmware_slot=0
106582025-04-30T06:44:13.007ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "a2a2ca50-2e9f-47ca-a988-381b5120d55b", "content-length": "170", "date": "Wed, 30 Apr 2025 06:44:12 GMT"} })
106592025-04-30T06:44:13.007ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0/component/rot/caboose?firmware_slot=1
106602025-04-30T06:44:13.011ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "1c5da0f5-a761-43cd-a19b-56f3aac3e1a5", "content-length": "170", "date": "Wed, 30 Apr 2025 06:44:12 GMT"} })
106612025-04-30T06:44:13.011ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0/component/stage0/caboose?firmware_slot=0
106622025-04-30T06:44:13.014ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "b6ced32f-5fc9-4b05-a0d1-6693fc9b2ad2", "content-length": "179", "date": "Wed, 30 Apr 2025 06:44:12 GMT"} })
106632025-04-30T06:44:13.015ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0/component/stage0/caboose?firmware_slot=1
106642025-04-30T06:44:13.017ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "765b3e1b-b247-473c-902b-efc6a7f4b9d7", "content-length": "179", "date": "Wed, 30 Apr 2025 06:44:12 GMT"} })
106652025-04-30T06:44:13.017ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0/component/rot/cmpa
106662025-04-30T06:44:13.018ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "eab23869-f124-4d67-bea5-39b39581c017", "content-length": "702", "date": "Wed, 30 Apr 2025 06:44:12 GMT"} })
106672025-04-30T06:44:13.019ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0/component/rot/cfpa
106682025-04-30T06:44:13.020ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "de75d93a-fdf3-49d6-ae1a-50a9ffa1a442", "content-length": "727", "date": "Wed, 30 Apr 2025 06:44:12 GMT"} })
106692025-04-30T06:44:13.020ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0/component/rot/cfpa
106702025-04-30T06:44:13.022ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "cb51c58f-5e6d-469b-8d05-18a7996e6abf", "content-length": "729", "date": "Wed, 30 Apr 2025 06:44:12 GMT"} })
106712025-04-30T06:44:13.022ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0/component/rot/cfpa
106722025-04-30T06:44:13.024ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "e7cdacf8-ecdd-45ed-b372-1859e8b07034", "content-length": "728", "date": "Wed, 30 Apr 2025 06:44:12 GMT"} })
106732025-04-30T06:44:13.024ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1
106742025-04-30T06:44:13.026ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1", status: 200, headers: {"content-type": "application/json", "x-request-id": "dd570e53-6d3c-418f-a544-3c39cd19a333", "content-length": "734", "date": "Wed, 30 Apr 2025 06:44:12 GMT"} })
106752025-04-30T06:44:13.026ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1/component/sp/caboose?firmware_slot=0
106762025-04-30T06:44:13.031ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "cbd4e9b4-232f-44cf-94b4-3bb5d4207eec", "content-length": "111", "date": "Wed, 30 Apr 2025 06:44:12 GMT"} })
106772025-04-30T06:44:13.031ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1/component/sp/caboose?firmware_slot=1
106782025-04-30T06:44:13.034ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "29e86ca9-7170-4e60-9f03-c768ce5af993", "content-length": "111", "date": "Wed, 30 Apr 2025 06:44:12 GMT"} })
106792025-04-30T06:44:13.034ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1/component/rot/caboose?firmware_slot=0
106802025-04-30T06:44:13.037ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "4d525116-f96f-4f65-9f6f-89163c3052cc", "content-length": "170", "date": "Wed, 30 Apr 2025 06:44:12 GMT"} })
106812025-04-30T06:44:13.037ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1/component/rot/caboose?firmware_slot=1
106822025-04-30T06:44:13.039ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "d544ccc4-a5dc-4e3b-8d11-4f4cc939b132", "content-length": "170", "date": "Wed, 30 Apr 2025 06:44:12 GMT"} })
106832025-04-30T06:44:13.039ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1/component/stage0/caboose?firmware_slot=0
106842025-04-30T06:44:13.042ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "88aab189-8d7e-4a22-9a1d-3e3e4081fbf7", "content-length": "179", "date": "Wed, 30 Apr 2025 06:44:12 GMT"} })
106852025-04-30T06:44:13.042ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1/component/stage0/caboose?firmware_slot=1
106862025-04-30T06:44:13.046ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "75854a3e-df8c-41c0-a060-6d8f8c4a3880", "content-length": "179", "date": "Wed, 30 Apr 2025 06:44:12 GMT"} })
106872025-04-30T06:44:13.046ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1/component/rot/cmpa
106882025-04-30T06:44:13.046ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "8a2515ac-1031-4a37-bcdc-441f405ee59a", "content-length": "702", "date": "Wed, 30 Apr 2025 06:44:12 GMT"} })
106892025-04-30T06:44:13.047ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1/component/rot/cfpa
106902025-04-30T06:44:13.048ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "62003124-4051-4dc8-8958-e24a0a45fd85", "content-length": "727", "date": "Wed, 30 Apr 2025 06:44:12 GMT"} })
106912025-04-30T06:44:13.048ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1/component/rot/cfpa
106922025-04-30T06:44:13.050ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "98183ed7-81b2-4d9f-82ca-28cb9c41c7ac", "content-length": "729", "date": "Wed, 30 Apr 2025 06:44:12 GMT"} })
106932025-04-30T06:44:13.050ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1/component/rot/cfpa
106942025-04-30T06:44:13.051ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "739f323a-1110-44d4-9cab-c5d9bc3f944b", "content-length": "728", "date": "Wed, 30 Apr 2025 06:44:12 GMT"} })
106952025-04-30T06:44:13.051ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0
106962025-04-30T06:44:13.052ZTRACtest_utilization (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory
106972025-04-30T06:44:13.052ZTRACtest_utilization (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
106982025-04-30T06:44:13.052ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
106992025-04-30T06:44:13.052ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
107002025-04-30T06:44:13.053ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
107012025-04-30T06:44:13.053ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "cf00e864-4f7e-4705-b89f-c27d6cc56163", "content-length": "723", "date": "Wed, 30 Apr 2025 06:44:12 GMT"} })
107022025-04-30T06:44:13.054ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0/component/sp/caboose?firmware_slot=0
107032025-04-30T06:44:13.056ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "2d55d40f-0df5-4354-8489-fcf7d63783dc", "content-length": "109", "date": "Wed, 30 Apr 2025 06:44:12 GMT"} })
107042025-04-30T06:44:13.056ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0/component/sp/caboose?firmware_slot=1
107052025-04-30T06:44:13.056ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
107062025-04-30T06:44:13.065ZDEBGtest_utilization (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory result = Ok(())
107072025-04-30T06:44:13.065ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
107082025-04-30T06:44:13.065ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
107092025-04-30T06:44:13.066ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
107102025-04-30T06:44:13.076ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
107112025-04-30T06:44:13.076ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
107122025-04-30T06:44:13.078ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
107132025-04-30T06:44:13.078ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "00c6a7bc-171f-478c-9881-4cdb0acd003c", "content-length": "109", "date": "Wed, 30 Apr 2025 06:44:12 GMT"} })
107142025-04-30T06:44:13.079ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0/component/rot/caboose?firmware_slot=0
107152025-04-30T06:44:13.084ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "468dd9b3-2846-4b28-b2cb-c40de2bc8534", "content-length": "169", "date": "Wed, 30 Apr 2025 06:44:12 GMT"} })
107162025-04-30T06:44:13.084ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0/component/rot/caboose?firmware_slot=1
107172025-04-30T06:44:13.090ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "30595477-a1ed-477d-8b02-64913ca84a6f", "content-length": "169", "date": "Wed, 30 Apr 2025 06:44:12 GMT"} })
107182025-04-30T06:44:13.090ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0/component/stage0/caboose?firmware_slot=0
107192025-04-30T06:44:13.095ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "c6213f47-d09a-4d6a-9560-8dcae625c5b4", "content-length": "178", "date": "Wed, 30 Apr 2025 06:44:12 GMT"} })
107202025-04-30T06:44:13.095ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0/component/stage0/caboose?firmware_slot=1
107212025-04-30T06:44:13.097ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "89f23e6b-c5d3-4949-8d42-3b57fe1929fc", "content-length": "178", "date": "Wed, 30 Apr 2025 06:44:12 GMT"} })
107222025-04-30T06:44:13.097ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0/component/rot/cmpa
107232025-04-30T06:44:13.100ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "d3d152f9-3741-48e4-b8b8-88b980214cd8", "content-length": "702", "date": "Wed, 30 Apr 2025 06:44:12 GMT"} })
107242025-04-30T06:44:13.102ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0/component/rot/cfpa
107252025-04-30T06:44:13.104ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "ae107e31-f640-4831-9ab7-c015f862a01f", "content-length": "727", "date": "Wed, 30 Apr 2025 06:44:12 GMT"} })
107262025-04-30T06:44:13.104ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0/component/rot/cfpa
107272025-04-30T06:44:13.108ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "cc82e1ab-ded5-418a-a29d-26ef51a11fa6", "content-length": "729", "date": "Wed, 30 Apr 2025 06:44:12 GMT"} })
107282025-04-30T06:44:13.108ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0/component/rot/cfpa
107292025-04-30T06:44:13.113ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "a9508093-d556-4c42-8e0b-e72357b896b1", "content-length": "728", "date": "Wed, 30 Apr 2025 06:44:12 GMT"} })
107302025-04-30T06:44:13.114ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1
107312025-04-30T06:44:13.120ZDEBGtest_utilization: attempt to fetch latest inventory collection result = Ok("found")
107322025-04-30T06:44:13.124ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1", status: 200, headers: {"content-type": "application/json", "x-request-id": "dde0afee-b50f-4057-9883-7e0281507981", "content-length": "723", "date": "Wed, 30 Apr 2025 06:44:12 GMT"} })
107332025-04-30T06:44:13.125ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1/component/sp/caboose?firmware_slot=0
107342025-04-30T06:44:13.130ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "6eb7903f-1f23-40dc-9a06-8ea8cbafe5ef", "content-length": "109", "date": "Wed, 30 Apr 2025 06:44:12 GMT"} })
107352025-04-30T06:44:13.131ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1/component/sp/caboose?firmware_slot=1
107362025-04-30T06:44:13.136ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "d1b57564-abd6-4f65-bee9-9dbca67abaa3", "content-length": "109", "date": "Wed, 30 Apr 2025 06:44:12 GMT"} })
107372025-04-30T06:44:13.137ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1/component/rot/caboose?firmware_slot=0
107382025-04-30T06:44:13.140ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "2b7b61e3-0264-4f24-bc7e-e36579bd45d9", "content-length": "169", "date": "Wed, 30 Apr 2025 06:44:12 GMT"} })
107392025-04-30T06:44:13.140ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1/component/rot/caboose?firmware_slot=1
107402025-04-30T06:44:13.145ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "7b92ea19-e300-4785-8fd4-fce3b5edf723", "content-length": "169", "date": "Wed, 30 Apr 2025 06:44:12 GMT"} })
107412025-04-30T06:44:13.145ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1/component/stage0/caboose?firmware_slot=0
107422025-04-30T06:44:13.149ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "069a9a1e-3f8e-4864-95a4-677eb08f5176", "content-length": "178", "date": "Wed, 30 Apr 2025 06:44:12 GMT"} })
107432025-04-30T06:44:13.149ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1/component/stage0/caboose?firmware_slot=1
107442025-04-30T06:44:13.152ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "f4d0f01d-189a-442c-8c77-a31b41574373", "content-length": "178", "date": "Wed, 30 Apr 2025 06:44:12 GMT"} })
107452025-04-30T06:44:13.153ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1/component/rot/cmpa
107462025-04-30T06:44:13.154ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "16e95c24-c567-499c-8655-8d3199791248", "content-length": "702", "date": "Wed, 30 Apr 2025 06:44:12 GMT"} })
107472025-04-30T06:44:13.154ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1/component/rot/cfpa
107482025-04-30T06:44:13.156ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "e12c7c0a-c739-4a5e-a9cc-a21fc747ddaf", "content-length": "727", "date": "Wed, 30 Apr 2025 06:44:12 GMT"} })
107492025-04-30T06:44:13.156ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1/component/rot/cfpa
107502025-04-30T06:44:13.157ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "41b33e3a-e36b-4e0b-a2ec-ab23b92c4e31", "content-length": "729", "date": "Wed, 30 Apr 2025 06:44:12 GMT"} })
107512025-04-30T06:44:13.157ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1/component/rot/cfpa
107522025-04-30T06:44:13.159ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "b785e44c-e7cb-4ab7-8b76-d37b1e038f04", "content-length": "728", "date": "Wed, 30 Apr 2025 06:44:12 GMT"} })
107532025-04-30T06:44:13.159ZTRACtest_utilization (BackgroundTasks): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet
107542025-04-30T06:44:13.159ZTRACtest_utilization (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
107552025-04-30T06:44:13.159ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
107562025-04-30T06:44:13.159ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
107572025-04-30T06:44:13.161ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
107582025-04-30T06:44:13.169ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
107592025-04-30T06:44:13.173ZDEBGtest_utilization (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet result = Ok(())
107602025-04-30T06:44:13.173ZTRACtest_utilization (BackgroundTasks): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet
107612025-04-30T06:44:13.174ZTRACtest_utilization (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
107622025-04-30T06:44:13.174ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
107632025-04-30T06:44:13.174ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
107642025-04-30T06:44:13.174ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
107652025-04-30T06:44:13.176ZTRACtest_utilization (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory
107662025-04-30T06:44:13.176ZTRACtest_utilization (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
107672025-04-30T06:44:13.176ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
107682025-04-30T06:44:13.176ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
107692025-04-30T06:44:13.177ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
107702025-04-30T06:44:13.177ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
107712025-04-30T06:44:13.182ZDEBGtest_utilization (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet result = Ok(())
107722025-04-30T06:44:13.182ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
107732025-04-30T06:44:13.182ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
107742025-04-30T06:44:13.182ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
107752025-04-30T06:44:13.185ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
107762025-04-30T06:44:13.193ZDEBGtest_utilization (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory result = Ok(())
107772025-04-30T06:44:13.193ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
107782025-04-30T06:44:13.193ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
107792025-04-30T06:44:13.194ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
107802025-04-30T06:44:13.267ZDEBGtest_utilization (BackgroundTasks): begin collection from Sled Agent background_task = inventory_collection sled_agent_url = http://[::1]:39365
107812025-04-30T06:44:13.267ZDEBGtest_utilization (BackgroundTasks): client request SledAgent = http://[::1]:39365 background_task = inventory_collection body = None method = GET uri = http://[::1]:39365/inventory
107822025-04-30T06:44:13.268ZINFOtest_utilization (dropshot): accepted connection local_addr = [::1]:39365 remote_addr = [::1]:41047 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
107832025-04-30T06:44:13.271ZTRACtest_utilization (dropshot): incoming request local_addr = [::1]:39365 method = GET remote_addr = [::1]:41047 req_id = d93aa2a1-4f85-4c11-8edc-302de9deb16c sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /inventory
107842025-04-30T06:44:13.273ZINFOtest_utilization (dropshot): request completed latency_us = 2573 local_addr = [::1]:39365 method = GET remote_addr = [::1]:41047 req_id = d93aa2a1-4f85-4c11-8edc-302de9deb16c response_code = 200 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /inventory
107852025-04-30T06:44:13.274ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
107862025-04-30T06:44:13.274ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
107872025-04-30T06:44:13.275ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
107882025-04-30T06:44:13.275ZDEBGtest_utilization (BackgroundTasks): client response SledAgent = http://[::1]:39365 background_task = inventory_collection result = Ok(Response { url: "http://[::1]:39365/inventory", status: 200, headers: {"content-type": "application/json", "x-request-id": "d93aa2a1-4f85-4c11-8edc-302de9deb16c", "content-length": "3141", "date": "Wed, 30 Apr 2025 06:44:12 GMT"} })
107892025-04-30T06:44:13.279ZDEBGtest_utilization (BackgroundTasks): begin collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
107902025-04-30T06:44:13.279ZDEBGtest_utilization (BackgroundTasks): end collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
107912025-04-30T06:44:13.279ZDEBGtest_utilization (BackgroundTasks): finished collection background_task = inventory_collection
107922025-04-30T06:44:13.281ZTRACtest_utilization (BackgroundTasks): authorize begin action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory
107932025-04-30T06:44:13.282ZTRACtest_utilization (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
107942025-04-30T06:44:13.282ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
107952025-04-30T06:44:13.282ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
107962025-04-30T06:44:13.283ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
107972025-04-30T06:44:13.288ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
107982025-04-30T06:44:13.295ZDEBGtest_utilization (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory result = Ok(())
107992025-04-30T06:44:13.295ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
108002025-04-30T06:44:13.295ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
108012025-04-30T06:44:13.296ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
108022025-04-30T06:44:13.326ZWARNtest_utilization (datastore): Non-retryable transaction failure err = DatabaseError(UniqueViolation, "duplicate key value violates unique constraint \\"inv_caboose_pkey\\"") transaction = inventory_insert_collection
108032025-04-30T06:44:13.327ZWARNtest_utilization (BackgroundTasks): inventory collection failed background_task = inventory_collection error = failed to collect inventory: saving inventory to database: Internal Error: unexpected database error: duplicate key value violates unique constraint "inv_caboose_pkey"
108042025-04-30T06:44:13.327ZDEBGtest_utilization (BackgroundTasks): activation complete background_task = inventory_collection elapsed = 1.114020793s iteration = 24 status = {"error":"failed to collect inventory: saving inventory to database: Internal Error: unexpected database error: duplicate key value violates unique constraint \\"inv_caboose_pkey\\""}
108052025-04-30T06:44:13.327ZDEBGtest_utilization (BackgroundTasks): activating background_task = inventory_collection iteration = 25 reason = Signaled
108062025-04-30T06:44:13.327ZTRACtest_utilization (BackgroundTasks): authorize begin action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory
108072025-04-30T06:44:13.327ZTRACtest_utilization (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
108082025-04-30T06:44:13.327ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
108092025-04-30T06:44:13.327ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
108102025-04-30T06:44:13.328ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
108112025-04-30T06:44:13.341ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
108122025-04-30T06:44:13.346ZDEBGtest_utilization (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory result = Ok(())
108132025-04-30T06:44:13.346ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
108142025-04-30T06:44:13.346ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
108152025-04-30T06:44:13.347ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
108162025-04-30T06:44:13.349ZDEBGtest_utilization: attempt to fetch latest inventory collection result = Ok("found")
108172025-04-30T06:44:13.353ZDEBGtest_utilization (BackgroundTasks): inventory_prune_one: nothing eligible for removal (too few) background_task = inventory_collection candidates = [(13d8b8d9-6c6b-47a6-9190-952544edb75b, 0)]
108182025-04-30T06:44:13.353ZTRACtest_utilization (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal
108192025-04-30T06:44:13.354ZTRACtest_utilization (dns): buffer buffer = Length: 67 (0x43) bytes\n0000: fa c3 01 00 00 01 00 00 00 00 00 01 04 5f 6d 67 ............._mg\n0010: 73 04 5f 74 63 70 0d 63 6f 6e 74 72 6f 6c 2d 70 s._tcp.control-p\n0020: 6c 61 6e 65 05 6f 78 69 64 65 08 69 6e 74 65 72 lane.oxide.inter\n0030: 6e 61 6c 00 00 21 00 01 00 00 29 04 d0 00 00 00 nal..!....).....\n0040: 00 00 00 ... kind = dns peer_addr = [::1]:63127 req_id = bb2ff739-3377-47cc-a2b4-00d17f63ee9b
108202025-04-30T06:44:13.354ZDEBGtest_utilization (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 64195,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_mgs._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_mgs._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 4,\n 95,\n 109,\n 103,\n 115,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:63127 req_id = bb2ff739-3377-47cc-a2b4-00d17f63ee9b
108212025-04-30T06:44:13.354ZDEBGtest_utilization (store): query key key = _mgs._tcp
108222025-04-30T06:44:13.354ZDEBGtest_utilization (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
108232025-04-30T06:44:13.354ZDEBGtest_utilization (dns): dns response additional_records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal"), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] kind = dns peer_addr = [::1]:63127 query = LowerQuery { name: LowerName(Name("_mgs._tcp.control-plane.oxide.internal.")), original: Query { name: Name("_mgs._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN } } records = [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 48759, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal") })) }] req_id = bb2ff739-3377-47cc-a2b4-00d17f63ee9b
108242025-04-30T06:44:13.355ZDEBGtest_utilization (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgs._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 48759, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }], valid_until: Instant { tv_sec: 4318, tv_nsec: 154042057 } })
108252025-04-30T06:44:13.355ZTRACtest_utilization (DnsResolver): lookup_service_targets: looking up SRV target
108262025-04-30T06:44:13.355ZTRACtest_utilization (dns): buffer buffer = Length: 108 (0x6c) bytes\n0000: e8 bc 01 00 00 01 00 00 00 00 00 01 2d 64 65 6e ............-den\n0010: 64 72 69 74 65 2d 62 36 64 36 35 33 34 31 2d 31 drite-b6d65341-1\n0020: 36 37 63 2d 34 31 64 66 2d 39 62 35 63 2d 34 31 67c-41df-9b5c-41\n0030: 63 64 65 64 39 39 63 32 32 39 04 68 6f 73 74 0d cded99c229.host.\n0040: 63 6f 6e 74 72 6f 6c 2d 70 6c 61 6e 65 05 6f 78 control-plane.ox\n0050: 69 64 65 08 69 6e 74 65 72 6e 61 6c 00 00 1c 00 ide.internal....\n0060: 01 00 00 29 04 d0 00 00 00 00 00 00 ...)........ kind = dns peer_addr = [::1]:61903 req_id = 0e710d3c-b163-499b-a2a1-1fa0ae04b949
108272025-04-30T06:44:13.355ZDEBGtest_utilization (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 59580,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n query_type: AAAA,\n query_class: IN,\n },\n },\n original: [\n 45,\n 100,\n 101,\n 110,\n 100,\n 114,\n 105,\n 116,\n 101,\n 45,\n 98,\n 54,\n 100,\n 54,\n 53,\n 51,\n 52,\n 49,\n 45,\n 49,\n 54,\n 55,\n 99,\n 45,\n 52,\n 49,\n 100,\n 102,\n 45,\n 57,\n 98,\n 53,\n 99,\n 45,\n 52,\n 49,\n 99,\n 100,\n 101,\n 100,\n 57,\n 57,\n 99,\n 50,\n 50,\n 57,\n 4,\n 104,\n 111,\n 115,\n 116,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 28,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:61903 req_id = 0e710d3c-b163-499b-a2a1-1fa0ae04b949
108282025-04-30T06:44:13.356ZDEBGtest_utilization (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
108292025-04-30T06:44:13.357ZDEBGtest_utilization (dns): dns response additional_records = [] kind = dns peer_addr = [::1]:61903 query = LowerQuery { name: LowerName(Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.")), original: Query { name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), query_type: AAAA, query_class: IN } } records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] req_id = 0e710d3c-b163-499b-a2a1-1fa0ae04b949
108302025-04-30T06:44:13.447ZTRACtest_utilization (DnsResolver): lookup_all_socket_v6 srv dns_name = _clickhouse-admin-keeper._tcp.control-plane.oxide.internal
108312025-04-30T06:44:13.448ZDEBGtest_utilization (results-sink-cluster): inserting 1 samples into database collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
108322025-04-30T06:44:13.448ZDEBGtest_utilization (results-sink): inserting 1 samples into database collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
108332025-04-30T06:44:13.448ZDEBGtest_utilization (clickhouse-client): unrolling 1 total samples collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 4d1485f0-110e-4161-9a33-c4610d68be5d
108342025-04-30T06:44:13.449ZTRACtest_utilization (clickhouse-client): inserting data collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 4d1485f0-110e-4161-9a33-c4610d68be5d n_columns = 4 n_rows = 2 sql = INSERT INTO oximeter.fields_string FORMAT Native
108352025-04-30T06:44:13.449ZTRACtest_utilization (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory
108362025-04-30T06:44:13.449ZTRACtest_utilization (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
108372025-04-30T06:44:13.449ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
108382025-04-30T06:44:13.449ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
108392025-04-30T06:44:13.451ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
108402025-04-30T06:44:13.452ZTRACtest_utilization (dns): buffer buffer = Length: 87 (0x57) bytes\n0000: 36 e5 01 00 00 01 00 00 00 00 00 01 18 5f 63 6c 6............_cl\n0010: 69 63 6b 68 6f 75 73 65 2d 61 64 6d 69 6e 2d 6b ickhouse-admin-k\n0020: 65 65 70 65 72 04 5f 74 63 70 0d 63 6f 6e 74 72 eeper._tcp.contr\n0030: 6f 6c 2d 70 6c 61 6e 65 05 6f 78 69 64 65 08 69 ol-plane.oxide.i\n0040: 6e 74 65 72 6e 61 6c 00 00 21 00 01 00 00 29 04 nternal..!....).\n0050: d0 00 00 00 00 00 00 ....... kind = dns peer_addr = [::1]:60221 req_id = 8946a450-b561-4d56-a304-2f8b45048fa4
108412025-04-30T06:44:13.452ZDEBGtest_utilization (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 14053,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 24,\n 95,\n 99,\n 108,\n 105,\n 99,\n 107,\n 104,\n 111,\n 117,\n 115,\n 101,\n 45,\n 97,\n 100,\n 109,\n 105,\n 110,\n 45,\n 107,\n 101,\n 101,\n 112,\n 101,\n 114,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:60221 req_id = 8946a450-b561-4d56-a304-2f8b45048fa4
108422025-04-30T06:44:13.452ZDEBGtest_utilization (store): query key key = _clickhouse-admin-keeper._tcp
108432025-04-30T06:44:13.453ZERROtest_utilization (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 14053, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 1, }, query: WireQuery { query: LowerQuery { name: LowerName( Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), ), original: Query { name: Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN, }, }, original: [ 24, 95, 99, 108, 105, 99, 107, 104, 111, 117, 115, 101, 45, 97, 100, 109, 105, 110, 45, 107, 101, 101, 112, 101, 114, 4, 95, 116, 99, 112, 13, 99, 111, 110, 116, 114, 111, 108, 45, 112, 108, 97, 110, 101, 5, 111, 120, 105, 100, 101, 8, 105, 110, 116, 101, 114, 110, 97, 108, 0, 0, 33, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: Some( Edns { rcode_high: 0, version: 0, dnssec_ok: false, max_payload: 1232, options: OPT { options: {}, }, }, ), } NXDOMAIN: no records found for name: "_clickhouse-admin-keeper._tcp.control-plane.oxide.internal." kind = dns peer_addr = [::1]:60221 req_id = 8946a450-b561-4d56-a304-2f8b45048fa4
108442025-04-30T06:44:13.453ZDEBGtest_utilization (BackgroundTasks): begin collection background_task = inventory_collection
108452025-04-30T06:44:13.453ZDEBGtest_utilization (BackgroundTasks): begin collection from MGS background_task = inventory_collection mgs_url = http://[::1]:48759
108462025-04-30T06:44:13.453ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/ignition
108472025-04-30T06:44:13.456ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
108482025-04-30T06:44:13.464ZDEBGtest_utilization (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory result = Ok(())
108492025-04-30T06:44:13.464ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
108502025-04-30T06:44:13.464ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
108512025-04-30T06:44:13.465ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
108522025-04-30T06:44:13.467ZDEBGtest_utilization (clickhouse-client): inserted rows into table collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 4d1485f0-110e-4161-9a33-c4610d68be5d n_rows = 2 table_name = fields_string
108532025-04-30T06:44:13.467ZTRACtest_utilization (clickhouse-client): inserting data collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 4d1485f0-110e-4161-9a33-c4610d68be5d n_columns = 4 n_rows = 1 sql = INSERT INTO oximeter.measurements_i64 FORMAT Native
108542025-04-30T06:44:13.473ZDEBGtest_utilization (clickhouse-client): inserted rows into table collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 4d1485f0-110e-4161-9a33-c4610d68be5d n_rows = 1 table_name = measurements_i64
108552025-04-30T06:44:13.474ZTRACtest_utilization (results-sink): successfully inserted samples collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
108562025-04-30T06:44:13.474ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/ignition", status: 200, headers: {"content-type": "application/json", "x-request-id": "3e1a9fd4-aae9-4681-818a-176a33a86e89", "content-length": "835", "date": "Wed, 30 Apr 2025 06:44:12 GMT"} })
108572025-04-30T06:44:13.474ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0
108582025-04-30T06:44:13.475ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
108592025-04-30T06:44:13.475ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
108602025-04-30T06:44:13.475ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
108612025-04-30T06:44:13.477ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "401d9305-6898-4845-ad17-958d9c9e9333", "content-length": "734", "date": "Wed, 30 Apr 2025 06:44:12 GMT"} })
108622025-04-30T06:44:13.477ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0/component/sp/caboose?firmware_slot=0
108632025-04-30T06:44:13.480ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "6e6ef4a2-7c82-4d59-a4d5-92772fde5a16", "content-length": "111", "date": "Wed, 30 Apr 2025 06:44:12 GMT"} })
108642025-04-30T06:44:13.480ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0/component/sp/caboose?firmware_slot=1
108652025-04-30T06:44:13.482ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "7f4aa0bb-30a7-4706-a852-1c1f4e7e5b36", "content-length": "111", "date": "Wed, 30 Apr 2025 06:44:12 GMT"} })
108662025-04-30T06:44:13.482ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0/component/rot/caboose?firmware_slot=0
108672025-04-30T06:44:13.486ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "569e019a-73f7-4bb6-815f-4e443e7a7e3f", "content-length": "170", "date": "Wed, 30 Apr 2025 06:44:12 GMT"} })
108682025-04-30T06:44:13.487ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0/component/rot/caboose?firmware_slot=1
108692025-04-30T06:44:13.489ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "a922ceef-ec16-41a9-9547-16629a8200bf", "content-length": "170", "date": "Wed, 30 Apr 2025 06:44:12 GMT"} })
108702025-04-30T06:44:13.489ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0/component/stage0/caboose?firmware_slot=0
108712025-04-30T06:44:13.492ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "5dd9db63-e0c6-48ce-9471-4ead027370fe", "content-length": "179", "date": "Wed, 30 Apr 2025 06:44:12 GMT"} })
108722025-04-30T06:44:13.492ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0/component/stage0/caboose?firmware_slot=1
108732025-04-30T06:44:13.509ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "5889be8f-c4f6-4ff4-8f3a-0e82556a9e73", "content-length": "179", "date": "Wed, 30 Apr 2025 06:44:12 GMT"} })
108742025-04-30T06:44:13.509ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0/component/rot/cmpa
108752025-04-30T06:44:13.511ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "e37f10df-58d6-4416-9e7a-8d06005c8f74", "content-length": "702", "date": "Wed, 30 Apr 2025 06:44:12 GMT"} })
108762025-04-30T06:44:13.512ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0/component/rot/cfpa
108772025-04-30T06:44:13.514ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "5c03133d-435b-4ae2-8d1b-158fc3b08632", "content-length": "727", "date": "Wed, 30 Apr 2025 06:44:12 GMT"} })
108782025-04-30T06:44:13.514ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0/component/rot/cfpa
108792025-04-30T06:44:13.518ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "1a9684ac-33b9-4776-b371-8e494362a7f1", "content-length": "729", "date": "Wed, 30 Apr 2025 06:44:12 GMT"} })
108802025-04-30T06:44:13.518ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0/component/rot/cfpa
108812025-04-30T06:44:13.521ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "c5c9d137-829d-46ec-a5bf-7d371ced87b6", "content-length": "728", "date": "Wed, 30 Apr 2025 06:44:12 GMT"} })
108822025-04-30T06:44:13.522ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1
108832025-04-30T06:44:13.534ZDEBGtest_utilization: attempt to fetch latest inventory collection result = Ok("found")
108842025-04-30T06:44:13.534ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1", status: 200, headers: {"content-type": "application/json", "x-request-id": "1c9dc276-f7a1-4dca-8770-c1a5aa8793e8", "content-length": "734", "date": "Wed, 30 Apr 2025 06:44:12 GMT"} })
108852025-04-30T06:44:13.535ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1/component/sp/caboose?firmware_slot=0
108862025-04-30T06:44:13.544ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "1ef3c0e9-3627-4a0f-abb5-42a7d76b638d", "content-length": "111", "date": "Wed, 30 Apr 2025 06:44:12 GMT"} })
108872025-04-30T06:44:13.544ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1/component/sp/caboose?firmware_slot=1
108882025-04-30T06:44:13.549ZWARNtest_utilization (results-sink-cluster): failed to insert some results into metric DB collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 error = Failed to check out connection to database: No backends found for this service file = oximeter/collector/src/results_sink.rs:92
108892025-04-30T06:44:13.550ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "07d2cf2c-f50a-4dda-8f07-ed7756aeefd8", "content-length": "111", "date": "Wed, 30 Apr 2025 06:44:12 GMT"} })
108902025-04-30T06:44:13.551ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1/component/rot/caboose?firmware_slot=0
108912025-04-30T06:44:13.556ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "2c2fe901-9f1c-4e10-863a-f15f801c1790", "content-length": "170", "date": "Wed, 30 Apr 2025 06:44:12 GMT"} })
108922025-04-30T06:44:13.556ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1/component/rot/caboose?firmware_slot=1
108932025-04-30T06:44:13.561ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "220c20db-37e2-4be5-b410-472d8ef15056", "content-length": "170", "date": "Wed, 30 Apr 2025 06:44:12 GMT"} })
108942025-04-30T06:44:13.561ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1/component/stage0/caboose?firmware_slot=0
108952025-04-30T06:44:13.565ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "491381fb-b309-4c1f-9731-a4d6c7517a76", "content-length": "179", "date": "Wed, 30 Apr 2025 06:44:12 GMT"} })
108962025-04-30T06:44:13.565ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1/component/stage0/caboose?firmware_slot=1
108972025-04-30T06:44:13.588ZTRACtest_utilization (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory
108982025-04-30T06:44:13.588ZTRACtest_utilization (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
108992025-04-30T06:44:13.588ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
109002025-04-30T06:44:13.588ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
109012025-04-30T06:44:13.589ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
109022025-04-30T06:44:13.601ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
109032025-04-30T06:44:13.613ZDEBGtest_utilization (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory result = Ok(())
109042025-04-30T06:44:13.613ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
109052025-04-30T06:44:13.613ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
109062025-04-30T06:44:13.613ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
109072025-04-30T06:44:13.626ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
109082025-04-30T06:44:13.626ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
109092025-04-30T06:44:13.629ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
109102025-04-30T06:44:13.630ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "697a8cc1-f4e9-480d-972f-ef8bb728e6a4", "content-length": "179", "date": "Wed, 30 Apr 2025 06:44:12 GMT"} })
109112025-04-30T06:44:13.631ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1/component/rot/cmpa
109122025-04-30T06:44:13.636ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "1d366062-f0b3-48ff-8aa2-7c6eb8a3b904", "content-length": "702", "date": "Wed, 30 Apr 2025 06:44:12 GMT"} })
109132025-04-30T06:44:13.636ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1/component/rot/cfpa
109142025-04-30T06:44:13.638ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "233a6998-9392-497c-893a-2a22f788b220", "content-length": "727", "date": "Wed, 30 Apr 2025 06:44:12 GMT"} })
109152025-04-30T06:44:13.638ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1/component/rot/cfpa
109162025-04-30T06:44:13.641ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "d873df78-8655-4ad2-9d07-8f3f10df1799", "content-length": "729", "date": "Wed, 30 Apr 2025 06:44:12 GMT"} })
109172025-04-30T06:44:13.641ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1/component/rot/cfpa
109182025-04-30T06:44:13.644ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "c8eea3ad-6462-4207-9a65-5ed320a5e6a5", "content-length": "728", "date": "Wed, 30 Apr 2025 06:44:12 GMT"} })
109192025-04-30T06:44:13.644ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0
109202025-04-30T06:44:13.646ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "e7322382-a105-458b-90e3-394376654694", "content-length": "723", "date": "Wed, 30 Apr 2025 06:44:12 GMT"} })
109212025-04-30T06:44:13.646ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0/component/sp/caboose?firmware_slot=0
109222025-04-30T06:44:13.651ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "3a23cf0e-e887-45ac-b517-e888486e57ec", "content-length": "109", "date": "Wed, 30 Apr 2025 06:44:12 GMT"} })
109232025-04-30T06:44:13.651ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0/component/sp/caboose?firmware_slot=1
109242025-04-30T06:44:13.654ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "7ed8a7f9-afaf-40be-8ab1-4a057d26eb5f", "content-length": "109", "date": "Wed, 30 Apr 2025 06:44:12 GMT"} })
109252025-04-30T06:44:13.654ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0/component/rot/caboose?firmware_slot=0
109262025-04-30T06:44:13.657ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "74730926-18fe-4261-a2ff-b9dff06ca052", "content-length": "169", "date": "Wed, 30 Apr 2025 06:44:12 GMT"} })
109272025-04-30T06:44:13.657ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0/component/rot/caboose?firmware_slot=1
109282025-04-30T06:44:13.661ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "e0bd1675-94e4-486d-9db3-57184675bbdf", "content-length": "169", "date": "Wed, 30 Apr 2025 06:44:12 GMT"} })
109292025-04-30T06:44:13.661ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0/component/stage0/caboose?firmware_slot=0
109302025-04-30T06:44:13.666ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "91964d03-f2b8-4db9-9539-b196b813e0f4", "content-length": "178", "date": "Wed, 30 Apr 2025 06:44:12 GMT"} })
109312025-04-30T06:44:13.666ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0/component/stage0/caboose?firmware_slot=1
109322025-04-30T06:44:13.668ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "09310e38-1dc7-47bb-b196-049dbd15baef", "content-length": "178", "date": "Wed, 30 Apr 2025 06:44:12 GMT"} })
109332025-04-30T06:44:13.668ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0/component/rot/cmpa
109342025-04-30T06:44:13.669ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "8e203c53-0156-478d-940f-2c774389d254", "content-length": "702", "date": "Wed, 30 Apr 2025 06:44:12 GMT"} })
109352025-04-30T06:44:13.670ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0/component/rot/cfpa
109362025-04-30T06:44:13.673ZDEBGtest_utilization: attempt to fetch latest inventory collection result = Ok("found")
109372025-04-30T06:44:13.673ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "83643751-3632-47f7-8c28-80247d666e75", "content-length": "727", "date": "Wed, 30 Apr 2025 06:44:12 GMT"} })
109382025-04-30T06:44:13.673ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0/component/rot/cfpa
109392025-04-30T06:44:13.676ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "473af003-0d2e-4ee5-afdf-811cc2fe7f47", "content-length": "729", "date": "Wed, 30 Apr 2025 06:44:12 GMT"} })
109402025-04-30T06:44:13.677ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0/component/rot/cfpa
109412025-04-30T06:44:13.680ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "762b1f10-bffa-4e55-9fe8-5f390a6b464f", "content-length": "728", "date": "Wed, 30 Apr 2025 06:44:12 GMT"} })
109422025-04-30T06:44:13.680ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1
109432025-04-30T06:44:13.682ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1", status: 200, headers: {"content-type": "application/json", "x-request-id": "41edc6ab-4019-49cf-a7f8-df49669b9d0f", "content-length": "723", "date": "Wed, 30 Apr 2025 06:44:12 GMT"} })
109442025-04-30T06:44:13.683ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1/component/sp/caboose?firmware_slot=0
109452025-04-30T06:44:13.687ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "f2bfbff8-9ca7-4a49-9583-f95becda747f", "content-length": "109", "date": "Wed, 30 Apr 2025 06:44:12 GMT"} })
109462025-04-30T06:44:13.688ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1/component/sp/caboose?firmware_slot=1
109472025-04-30T06:44:13.693ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "55e05ae5-f36d-4569-a1f5-629c22f91b83", "content-length": "109", "date": "Wed, 30 Apr 2025 06:44:12 GMT"} })
109482025-04-30T06:44:13.693ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1/component/rot/caboose?firmware_slot=0
109492025-04-30T06:44:13.698ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "0ac700c2-11c7-4ff5-bb20-4415668bdee0", "content-length": "169", "date": "Wed, 30 Apr 2025 06:44:12 GMT"} })
109502025-04-30T06:44:13.699ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1/component/rot/caboose?firmware_slot=1
109512025-04-30T06:44:13.700ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "7db5c23a-43a2-4554-b881-028bb8477455", "content-length": "169", "date": "Wed, 30 Apr 2025 06:44:12 GMT"} })
109522025-04-30T06:44:13.700ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1/component/stage0/caboose?firmware_slot=0
109532025-04-30T06:44:13.704ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "ac57a21f-bb43-41cf-9734-ae56c8e57fee", "content-length": "178", "date": "Wed, 30 Apr 2025 06:44:12 GMT"} })
109542025-04-30T06:44:13.704ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1/component/stage0/caboose?firmware_slot=1
109552025-04-30T06:44:13.708ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "5f6e46dc-f32b-462d-ac49-0f597c96d44d", "content-length": "178", "date": "Wed, 30 Apr 2025 06:44:12 GMT"} })
109562025-04-30T06:44:13.709ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1/component/rot/cmpa
109572025-04-30T06:44:13.710ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "69e0e96e-9394-4cf0-ac74-684ce2a279a6", "content-length": "702", "date": "Wed, 30 Apr 2025 06:44:12 GMT"} })
109582025-04-30T06:44:13.711ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1/component/rot/cfpa
109592025-04-30T06:44:13.714ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "8a115e5f-47a8-4e4c-b572-a4a9410732e7", "content-length": "727", "date": "Wed, 30 Apr 2025 06:44:12 GMT"} })
109602025-04-30T06:44:13.714ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1/component/rot/cfpa
109612025-04-30T06:44:13.715ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "1399da02-dd98-4ed3-932c-6d8be4ca394d", "content-length": "729", "date": "Wed, 30 Apr 2025 06:44:12 GMT"} })
109622025-04-30T06:44:13.716ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1/component/rot/cfpa
109632025-04-30T06:44:13.718ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "4eee94ee-86eb-4d7d-9218-41d3651e8c53", "content-length": "728", "date": "Wed, 30 Apr 2025 06:44:12 GMT"} })
109642025-04-30T06:44:13.718ZTRACtest_utilization (BackgroundTasks): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet
109652025-04-30T06:44:13.718ZTRACtest_utilization (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
109662025-04-30T06:44:13.718ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
109672025-04-30T06:44:13.719ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
109682025-04-30T06:44:13.719ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
109692025-04-30T06:44:13.722ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
109702025-04-30T06:44:13.727ZDEBGtest_utilization (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet result = Ok(())
109712025-04-30T06:44:13.727ZTRACtest_utilization (BackgroundTasks): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet
109722025-04-30T06:44:13.727ZTRACtest_utilization (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
109732025-04-30T06:44:13.727ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
109742025-04-30T06:44:13.727ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
109752025-04-30T06:44:13.727ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
109762025-04-30T06:44:13.728ZTRACtest_utilization (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory
109772025-04-30T06:44:13.728ZTRACtest_utilization (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
109782025-04-30T06:44:13.728ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
109792025-04-30T06:44:13.728ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
109802025-04-30T06:44:13.728ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
109812025-04-30T06:44:13.730ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
109822025-04-30T06:44:13.737ZDEBGtest_utilization (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet result = Ok(())
109832025-04-30T06:44:13.737ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
109842025-04-30T06:44:13.737ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
109852025-04-30T06:44:13.737ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
109862025-04-30T06:44:13.740ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
109872025-04-30T06:44:13.745ZDEBGtest_utilization (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory result = Ok(())
109882025-04-30T06:44:13.745ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
109892025-04-30T06:44:13.745ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
109902025-04-30T06:44:13.746ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
109912025-04-30T06:44:13.820ZDEBGtest_utilization (BackgroundTasks): begin collection from Sled Agent background_task = inventory_collection sled_agent_url = http://[::1]:39365
109922025-04-30T06:44:13.820ZDEBGtest_utilization (BackgroundTasks): client request SledAgent = http://[::1]:39365 background_task = inventory_collection body = None method = GET uri = http://[::1]:39365/inventory
109932025-04-30T06:44:13.821ZINFOtest_utilization (dropshot): accepted connection local_addr = [::1]:39365 remote_addr = [::1]:40472 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
109942025-04-30T06:44:13.822ZTRACtest_utilization (dropshot): incoming request local_addr = [::1]:39365 method = GET remote_addr = [::1]:40472 req_id = 9c0be43b-f337-4fb0-baa2-e1208a2eeba7 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /inventory
109952025-04-30T06:44:13.823ZINFOtest_utilization (dropshot): request completed latency_us = 598 local_addr = [::1]:39365 method = GET remote_addr = [::1]:40472 req_id = 9c0be43b-f337-4fb0-baa2-e1208a2eeba7 response_code = 200 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /inventory
109962025-04-30T06:44:13.823ZDEBGtest_utilization (BackgroundTasks): client response SledAgent = http://[::1]:39365 background_task = inventory_collection result = Ok(Response { url: "http://[::1]:39365/inventory", status: 200, headers: {"content-type": "application/json", "x-request-id": "9c0be43b-f337-4fb0-baa2-e1208a2eeba7", "content-length": "3141", "date": "Wed, 30 Apr 2025 06:44:13 GMT"} })
109972025-04-30T06:44:13.828ZDEBGtest_utilization (BackgroundTasks): begin collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
109982025-04-30T06:44:13.828ZDEBGtest_utilization (BackgroundTasks): end collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
109992025-04-30T06:44:13.828ZDEBGtest_utilization (BackgroundTasks): finished collection background_task = inventory_collection
110002025-04-30T06:44:13.832ZTRACtest_utilization (BackgroundTasks): authorize begin action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory
110012025-04-30T06:44:13.832ZTRACtest_utilization (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
110022025-04-30T06:44:13.832ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
110032025-04-30T06:44:13.833ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
110042025-04-30T06:44:13.833ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
110052025-04-30T06:44:13.834ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
110062025-04-30T06:44:13.834ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
110072025-04-30T06:44:13.844ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
110082025-04-30T06:44:13.847ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
110092025-04-30T06:44:13.851ZDEBGtest_utilization (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory result = Ok(())
110102025-04-30T06:44:13.852ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
110112025-04-30T06:44:13.852ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
110122025-04-30T06:44:13.852ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
110132025-04-30T06:44:13.889ZWARNtest_utilization (datastore): Non-retryable transaction failure err = DatabaseError(UniqueViolation, "duplicate key value violates unique constraint \\"inv_caboose_pkey\\"") transaction = inventory_insert_collection
110142025-04-30T06:44:13.890ZWARNtest_utilization (BackgroundTasks): inventory collection failed background_task = inventory_collection error = failed to collect inventory: saving inventory to database: Internal Error: unexpected database error: duplicate key value violates unique constraint "inv_caboose_pkey"
110152025-04-30T06:44:13.890ZDEBGtest_utilization (BackgroundTasks): activation complete background_task = inventory_collection elapsed = 562.574723ms iteration = 25 status = {"error":"failed to collect inventory: saving inventory to database: Internal Error: unexpected database error: duplicate key value violates unique constraint \\"inv_caboose_pkey\\""}
110162025-04-30T06:44:13.890ZDEBGtest_utilization (BackgroundTasks): activating background_task = inventory_collection iteration = 26 reason = Signaled
110172025-04-30T06:44:13.890ZTRACtest_utilization (BackgroundTasks): authorize begin action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory
110182025-04-30T06:44:13.890ZTRACtest_utilization (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
110192025-04-30T06:44:13.890ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
110202025-04-30T06:44:13.891ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
110212025-04-30T06:44:13.892ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
110222025-04-30T06:44:13.893ZDEBGtest_utilization: attempt to fetch latest inventory collection result = Ok("found")
110232025-04-30T06:44:13.896ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
110242025-04-30T06:44:13.902ZDEBGtest_utilization (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory result = Ok(())
110252025-04-30T06:44:13.902ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
110262025-04-30T06:44:13.902ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
110272025-04-30T06:44:13.902ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
110282025-04-30T06:44:13.905ZDEBGtest_utilization (BackgroundTasks): inventory_prune_one: nothing eligible for removal (too few) background_task = inventory_collection candidates = [(13d8b8d9-6c6b-47a6-9190-952544edb75b, 0)]
110292025-04-30T06:44:13.905ZTRACtest_utilization (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal
110302025-04-30T06:44:13.907ZTRACtest_utilization (dns): buffer buffer = Length: 67 (0x43) bytes\n0000: 73 57 01 00 00 01 00 00 00 00 00 01 04 5f 6d 67 sW..........._mg\n0010: 73 04 5f 74 63 70 0d 63 6f 6e 74 72 6f 6c 2d 70 s._tcp.control-p\n0020: 6c 61 6e 65 05 6f 78 69 64 65 08 69 6e 74 65 72 lane.oxide.inter\n0030: 6e 61 6c 00 00 21 00 01 00 00 29 04 d0 00 00 00 nal..!....).....\n0040: 00 00 00 ... kind = dns peer_addr = [::1]:62628 req_id = 8f5184ee-4416-495c-a92c-af9242cc9de9
110312025-04-30T06:44:13.907ZDEBGtest_utilization (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 29527,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_mgs._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_mgs._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 4,\n 95,\n 109,\n 103,\n 115,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:62628 req_id = 8f5184ee-4416-495c-a92c-af9242cc9de9
110322025-04-30T06:44:13.907ZDEBGtest_utilization (store): query key key = _mgs._tcp
110332025-04-30T06:44:13.907ZDEBGtest_utilization (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
110342025-04-30T06:44:13.908ZDEBGtest_utilization (dns): dns response additional_records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal"), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] kind = dns peer_addr = [::1]:62628 query = LowerQuery { name: LowerName(Name("_mgs._tcp.control-plane.oxide.internal.")), original: Query { name: Name("_mgs._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN } } records = [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 48759, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal") })) }] req_id = 8f5184ee-4416-495c-a92c-af9242cc9de9
110352025-04-30T06:44:13.909ZDEBGtest_utilization (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgs._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 48759, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }], valid_until: Instant { tv_sec: 4318, tv_nsec: 707965813 } })
110362025-04-30T06:44:13.909ZTRACtest_utilization (DnsResolver): lookup_service_targets: looking up SRV target
110372025-04-30T06:44:13.909ZTRACtest_utilization (dns): buffer buffer = Length: 108 (0x6c) bytes\n0000: 44 00 01 00 00 01 00 00 00 00 00 01 2d 64 65 6e D...........-den\n0010: 64 72 69 74 65 2d 62 36 64 36 35 33 34 31 2d 31 drite-b6d65341-1\n0020: 36 37 63 2d 34 31 64 66 2d 39 62 35 63 2d 34 31 67c-41df-9b5c-41\n0030: 63 64 65 64 39 39 63 32 32 39 04 68 6f 73 74 0d cded99c229.host.\n0040: 63 6f 6e 74 72 6f 6c 2d 70 6c 61 6e 65 05 6f 78 control-plane.ox\n0050: 69 64 65 08 69 6e 74 65 72 6e 61 6c 00 00 1c 00 ide.internal....\n0060: 01 00 00 29 04 d0 00 00 00 00 00 00 ...)........ kind = dns peer_addr = [::1]:59814 req_id = 4ae6380d-6c45-42c2-aedc-77ed514452a4
110382025-04-30T06:44:13.909ZDEBGtest_utilization (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 17408,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n query_type: AAAA,\n query_class: IN,\n },\n },\n original: [\n 45,\n 100,\n 101,\n 110,\n 100,\n 114,\n 105,\n 116,\n 101,\n 45,\n 98,\n 54,\n 100,\n 54,\n 53,\n 51,\n 52,\n 49,\n 45,\n 49,\n 54,\n 55,\n 99,\n 45,\n 52,\n 49,\n 100,\n 102,\n 45,\n 57,\n 98,\n 53,\n 99,\n 45,\n 52,\n 49,\n 99,\n 100,\n 101,\n 100,\n 57,\n 57,\n 99,\n 50,\n 50,\n 57,\n 4,\n 104,\n 111,\n 115,\n 116,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 28,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:59814 req_id = 4ae6380d-6c45-42c2-aedc-77ed514452a4
110392025-04-30T06:44:13.910ZDEBGtest_utilization (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
110402025-04-30T06:44:13.910ZDEBGtest_utilization (dns): dns response additional_records = [] kind = dns peer_addr = [::1]:59814 query = LowerQuery { name: LowerName(Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.")), original: Query { name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), query_type: AAAA, query_class: IN } } records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] req_id = 4ae6380d-6c45-42c2-aedc-77ed514452a4
110412025-04-30T06:44:14.004ZTRACtest_utilization (DnsResolver): lookup_all_socket_v6 srv dns_name = _clickhouse-admin-keeper._tcp.control-plane.oxide.internal
110422025-04-30T06:44:14.005ZTRACtest_utilization (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory
110432025-04-30T06:44:14.006ZTRACtest_utilization (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
110442025-04-30T06:44:14.006ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
110452025-04-30T06:44:14.006ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
110462025-04-30T06:44:14.007ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
110472025-04-30T06:44:14.008ZINFOtest_utilization (oximeter-agent): refreshing list of producers from Nexus collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 file = oximeter/collector/src/agent.rs:562
110482025-04-30T06:44:14.008ZDEBGtest_utilization (collection-task): sent timer-based collection request to the collection task collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
110492025-04-30T06:44:14.008ZDEBGtest_utilization (collection-task): collecting from producer collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
110502025-04-30T06:44:14.009ZDEBGtest_utilization (oximeter): client request body = None method = GET uri = http://127.0.0.1:32956/metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?sort_by=id_ascending
110512025-04-30T06:44:14.010ZTRACtest_utilization (dropshot_internal): incoming request local_addr = 127.0.0.1:32956 method = GET remote_addr = 127.0.0.1:55797 req_id = 8476b957-2a34-4acf-bf48-42df32e7dc89 uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?sort_by=id_ascending
110522025-04-30T06:44:14.011ZTRACtest_utilization (dns): buffer buffer = Length: 87 (0x57) bytes\n0000: 2c fc 01 00 00 01 00 00 00 00 00 01 18 5f 63 6c ,............_cl\n0010: 69 63 6b 68 6f 75 73 65 2d 61 64 6d 69 6e 2d 6b ickhouse-admin-k\n0020: 65 65 70 65 72 04 5f 74 63 70 0d 63 6f 6e 74 72 eeper._tcp.contr\n0030: 6f 6c 2d 70 6c 61 6e 65 05 6f 78 69 64 65 08 69 ol-plane.oxide.i\n0040: 6e 74 65 72 6e 61 6c 00 00 21 00 01 00 00 29 04 nternal..!....).\n0050: d0 00 00 00 00 00 00 ....... kind = dns peer_addr = [::1]:49970 req_id = ae6428d4-7ac0-4bbb-8bbf-c4e811af6986
110532025-04-30T06:44:14.011ZDEBGtest_utilization (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 11516,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 24,\n 95,\n 99,\n 108,\n 105,\n 99,\n 107,\n 104,\n 111,\n 117,\n 115,\n 101,\n 45,\n 97,\n 100,\n 109,\n 105,\n 110,\n 45,\n 107,\n 101,\n 101,\n 112,\n 101,\n 114,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:49970 req_id = ae6428d4-7ac0-4bbb-8bbf-c4e811af6986
110542025-04-30T06:44:14.011ZDEBGtest_utilization (store): query key key = _clickhouse-admin-keeper._tcp
110552025-04-30T06:44:14.012ZERROtest_utilization (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 11516, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 1, }, query: WireQuery { query: LowerQuery { name: LowerName( Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), ), original: Query { name: Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN, }, }, original: [ 24, 95, 99, 108, 105, 99, 107, 104, 111, 117, 115, 101, 45, 97, 100, 109, 105, 110, 45, 107, 101, 101, 112, 101, 114, 4, 95, 116, 99, 112, 13, 99, 111, 110, 116, 114, 111, 108, 45, 112, 108, 97, 110, 101, 5, 111, 120, 105, 100, 101, 8, 105, 110, 116, 101, 114, 110, 97, 108, 0, 0, 33, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: Some( Edns { rcode_high: 0, version: 0, dnssec_ok: false, max_payload: 1232, options: OPT { options: {}, }, }, ), } NXDOMAIN: no records found for name: "_clickhouse-admin-keeper._tcp.control-plane.oxide.internal." kind = dns peer_addr = [::1]:49970 req_id = ae6428d4-7ac0-4bbb-8bbf-c4e811af6986
110562025-04-30T06:44:14.012ZTRACtest_utilization (dropshot_internal): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = 127.0.0.1:32956 method = GET remote_addr = 127.0.0.1:55797 req_id = 8476b957-2a34-4acf-bf48-42df32e7dc89 resource = Database uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?sort_by=id_ascending
110572025-04-30T06:44:14.012ZDEBGtest_utilization (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = 127.0.0.1:32956 method = GET remote_addr = 127.0.0.1:55797 req_id = 8476b957-2a34-4acf-bf48-42df32e7dc89 roles = RoleSet { roles: {} } uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?sort_by=id_ascending
110582025-04-30T06:44:14.013ZDEBGtest_utilization (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = 127.0.0.1:32956 method = GET remote_addr = 127.0.0.1:55797 req_id = 8476b957-2a34-4acf-bf48-42df32e7dc89 resource = Database result = Ok(()) uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?sort_by=id_ascending
110592025-04-30T06:44:14.016ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
110602025-04-30T06:44:14.029ZDEBGtest_utilization (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory result = Ok(())
110612025-04-30T06:44:14.029ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
110622025-04-30T06:44:14.029ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
110632025-04-30T06:44:14.031ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
110642025-04-30T06:44:14.033ZDEBGtest_utilization (BackgroundTasks): begin collection background_task = inventory_collection
110652025-04-30T06:44:14.033ZDEBGtest_utilization (BackgroundTasks): begin collection from MGS background_task = inventory_collection mgs_url = http://[::1]:48759
110662025-04-30T06:44:14.033ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/ignition
110672025-04-30T06:44:14.034ZTRACtest_utilization (collection-task): sent collection request to producer collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
110682025-04-30T06:44:14.035ZDEBGtest_utilization (collection-task): collected results from producer collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 n_results = 1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
110692025-04-30T06:44:14.035ZTRACtest_utilization (collection-task): forwarded results to main collection loop collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
110702025-04-30T06:44:14.035ZTRACtest_utilization (collection-task): top of inner collection loop, waiting for next request collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
110712025-04-30T06:44:14.044ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/ignition", status: 200, headers: {"content-type": "application/json", "x-request-id": "17837b0e-1005-40ef-b1c7-8dc5142611f0", "content-length": "835", "date": "Wed, 30 Apr 2025 06:44:13 GMT"} })
110722025-04-30T06:44:14.044ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0
110732025-04-30T06:44:14.045ZINFOtest_utilization (dropshot_internal): request completed latency_us = 34960 local_addr = 127.0.0.1:32956 method = GET remote_addr = 127.0.0.1:55797 req_id = 8476b957-2a34-4acf-bf48-42df32e7dc89 response_code = 200 uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?sort_by=id_ascending
110742025-04-30T06:44:14.045ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
110752025-04-30T06:44:14.045ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
110762025-04-30T06:44:14.047ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
110772025-04-30T06:44:14.047ZDEBGtest_utilization (oximeter): client response result = Ok(Response { url: "http://127.0.0.1:32956/metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?sort_by=id_ascending", status: 200, headers: {"content-type": "application/json", "x-request-id": "8476b957-2a34-4acf-bf48-42df32e7dc89", "content-length": "405", "date": "Wed, 30 Apr 2025 06:44:13 GMT"} })
110782025-04-30T06:44:14.047ZDEBGtest_utilization (oximeter): client request body = None method = GET uri = http://127.0.0.1:32956/metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJhYzU1ZDhkOS1hMjk4LTQ1MzQtODk4Yi1iZWVlMWM5ZTRlNTIifX0%3D
110792025-04-30T06:44:14.048ZTRACtest_utilization (dropshot_internal): incoming request local_addr = 127.0.0.1:32956 method = GET remote_addr = 127.0.0.1:55797 req_id = 32b5ad53-b066-4949-ba65-62830c09e7ec uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJhYzU1ZDhkOS1hMjk4LTQ1MzQtODk4Yi1iZWVlMWM5ZTRlNTIifX0%3D
110802025-04-30T06:44:14.048ZTRACtest_utilization (dropshot_internal): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = 127.0.0.1:32956 method = GET remote_addr = 127.0.0.1:55797 req_id = 32b5ad53-b066-4949-ba65-62830c09e7ec resource = Database uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJhYzU1ZDhkOS1hMjk4LTQ1MzQtODk4Yi1iZWVlMWM5ZTRlNTIifX0%3D
110812025-04-30T06:44:14.049ZDEBGtest_utilization (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = 127.0.0.1:32956 method = GET remote_addr = 127.0.0.1:55797 req_id = 32b5ad53-b066-4949-ba65-62830c09e7ec roles = RoleSet { roles: {} } uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJhYzU1ZDhkOS1hMjk4LTQ1MzQtODk4Yi1iZWVlMWM5ZTRlNTIifX0%3D
110822025-04-30T06:44:14.049ZDEBGtest_utilization (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = 127.0.0.1:32956 method = GET remote_addr = 127.0.0.1:55797 req_id = 32b5ad53-b066-4949-ba65-62830c09e7ec resource = Database result = Ok(()) uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJhYzU1ZDhkOS1hMjk4LTQ1MzQtODk4Yi1iZWVlMWM5ZTRlNTIifX0%3D
110832025-04-30T06:44:14.051ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "1f0fa26f-1e54-4f2f-96c1-c7cb0fc881b4", "content-length": "734", "date": "Wed, 30 Apr 2025 06:44:13 GMT"} })
110842025-04-30T06:44:14.052ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0/component/sp/caboose?firmware_slot=0
110852025-04-30T06:44:14.052ZINFOtest_utilization (dropshot_internal): request completed latency_us = 3916 local_addr = 127.0.0.1:32956 method = GET remote_addr = 127.0.0.1:55797 req_id = 32b5ad53-b066-4949-ba65-62830c09e7ec response_code = 200 uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJhYzU1ZDhkOS1hMjk4LTQ1MzQtODk4Yi1iZWVlMWM5ZTRlNTIifX0%3D
110862025-04-30T06:44:14.053ZDEBGtest_utilization (oximeter): client response result = Ok(Response { url: "http://127.0.0.1:32956/metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJhYzU1ZDhkOS1hMjk4LTQ1MzQtODk4Yi1iZWVlMWM5ZTRlNTIifX0%3D", status: 200, headers: {"content-type": "application/json", "x-request-id": "32b5ad53-b066-4949-ba65-62830c09e7ec", "content-length": "29", "date": "Wed, 30 Apr 2025 06:44:13 GMT"} })
110872025-04-30T06:44:14.053ZTRACtest_utilization (oximeter-agent): ignoring request to update existing metric producer, since the endpoint information is the same as the existing collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
110882025-04-30T06:44:14.053ZTRACtest_utilization (oximeter-agent): ignoring request to update existing metric producer, since the endpoint information is the same as the existing collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = ac55d8d9-a298-4534-898b-beee1c9e4e52
110892025-04-30T06:44:14.053ZINFOtest_utilization (oximeter-agent): refreshed list of producers from Nexus collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 file = oximeter/collector/src/agent.rs:637 n_current_tasks = 2 n_pruned_tasks = 0
110902025-04-30T06:44:14.054ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "60f847f4-79e7-4e28-bae4-2c9f110f022f", "content-length": "111", "date": "Wed, 30 Apr 2025 06:44:13 GMT"} })
110912025-04-30T06:44:14.054ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0/component/sp/caboose?firmware_slot=1
110922025-04-30T06:44:14.056ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "d0b86616-16c4-4d07-8127-d660cc74ce31", "content-length": "111", "date": "Wed, 30 Apr 2025 06:44:13 GMT"} })
110932025-04-30T06:44:14.056ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0/component/rot/caboose?firmware_slot=0
110942025-04-30T06:44:14.058ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "2d020d51-c19f-46aa-a14c-99dcc0f946ff", "content-length": "170", "date": "Wed, 30 Apr 2025 06:44:13 GMT"} })
110952025-04-30T06:44:14.058ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0/component/rot/caboose?firmware_slot=1
110962025-04-30T06:44:14.061ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "115c8653-7188-42aa-a0f2-7546acf3e511", "content-length": "170", "date": "Wed, 30 Apr 2025 06:44:13 GMT"} })
110972025-04-30T06:44:14.061ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0/component/stage0/caboose?firmware_slot=0
110982025-04-30T06:44:14.064ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "a760c472-a6b3-4f75-80c7-279ceb93e24e", "content-length": "179", "date": "Wed, 30 Apr 2025 06:44:13 GMT"} })
110992025-04-30T06:44:14.064ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0/component/stage0/caboose?firmware_slot=1
111002025-04-30T06:44:14.067ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "7334da0b-5b3a-4a3e-aa36-6b7edc5bd828", "content-length": "179", "date": "Wed, 30 Apr 2025 06:44:13 GMT"} })
111012025-04-30T06:44:14.067ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0/component/rot/cmpa
111022025-04-30T06:44:14.069ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "1020ac0f-564e-47fb-8125-ac30e80f2609", "content-length": "702", "date": "Wed, 30 Apr 2025 06:44:13 GMT"} })
111032025-04-30T06:44:14.070ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0/component/rot/cfpa
111042025-04-30T06:44:14.073ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "4f9df570-c071-4059-bdf1-a3fc0a208f36", "content-length": "727", "date": "Wed, 30 Apr 2025 06:44:13 GMT"} })
111052025-04-30T06:44:14.073ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0/component/rot/cfpa
111062025-04-30T06:44:14.077ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "e019050b-13a9-4ca5-b91b-628ea98e0d87", "content-length": "729", "date": "Wed, 30 Apr 2025 06:44:13 GMT"} })
111072025-04-30T06:44:14.077ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0/component/rot/cfpa
111082025-04-30T06:44:14.077ZDEBGtest_utilization: attempt to fetch latest inventory collection result = Ok("found")
111092025-04-30T06:44:14.078ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "89f388d1-8a42-41b3-92e0-126cd7ee10e6", "content-length": "728", "date": "Wed, 30 Apr 2025 06:44:13 GMT"} })
111102025-04-30T06:44:14.078ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1
111112025-04-30T06:44:14.081ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1", status: 200, headers: {"content-type": "application/json", "x-request-id": "2f320a6a-789b-4942-aecb-58c8fe329fab", "content-length": "734", "date": "Wed, 30 Apr 2025 06:44:13 GMT"} })
111122025-04-30T06:44:14.081ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1/component/sp/caboose?firmware_slot=0
111132025-04-30T06:44:14.084ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "e9d28739-66e2-44ae-ace3-7f5b04d9e223", "content-length": "111", "date": "Wed, 30 Apr 2025 06:44:13 GMT"} })
111142025-04-30T06:44:14.084ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1/component/sp/caboose?firmware_slot=1
111152025-04-30T06:44:14.086ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "2ca5ba78-8c9b-4321-a936-7cae3040c657", "content-length": "111", "date": "Wed, 30 Apr 2025 06:44:13 GMT"} })
111162025-04-30T06:44:14.086ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1/component/rot/caboose?firmware_slot=0
111172025-04-30T06:44:14.088ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "29287f24-7d9c-44fe-bf66-04370e81f1b5", "content-length": "170", "date": "Wed, 30 Apr 2025 06:44:13 GMT"} })
111182025-04-30T06:44:14.088ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1/component/rot/caboose?firmware_slot=1
111192025-04-30T06:44:14.092ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "e4ee700c-550f-418e-bedd-a6c9fcdd3230", "content-length": "170", "date": "Wed, 30 Apr 2025 06:44:13 GMT"} })
111202025-04-30T06:44:14.092ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1/component/stage0/caboose?firmware_slot=0
111212025-04-30T06:44:14.097ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "c8c5f715-2743-4c0d-9a95-fdb92d40aa10", "content-length": "179", "date": "Wed, 30 Apr 2025 06:44:13 GMT"} })
111222025-04-30T06:44:14.098ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1/component/stage0/caboose?firmware_slot=1
111232025-04-30T06:44:14.103ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "7b7b259f-02ac-4c4a-9acd-2762c35914ac", "content-length": "179", "date": "Wed, 30 Apr 2025 06:44:13 GMT"} })
111242025-04-30T06:44:14.105ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1/component/rot/cmpa
111252025-04-30T06:44:14.107ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "079311bf-e2be-4e23-90f7-6f5f876744b4", "content-length": "702", "date": "Wed, 30 Apr 2025 06:44:13 GMT"} })
111262025-04-30T06:44:14.107ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1/component/rot/cfpa
111272025-04-30T06:44:14.109ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "a4a787f3-60c1-46a8-8a8a-1900e95bad64", "content-length": "727", "date": "Wed, 30 Apr 2025 06:44:13 GMT"} })
111282025-04-30T06:44:14.109ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1/component/rot/cfpa
111292025-04-30T06:44:14.116ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "1edc316b-3bc5-406c-9916-72f11bcafeb4", "content-length": "729", "date": "Wed, 30 Apr 2025 06:44:13 GMT"} })
111302025-04-30T06:44:14.116ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1/component/rot/cfpa
111312025-04-30T06:44:14.117ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "ceec12c1-90f0-4cdb-9ea6-1dd82db0dc6d", "content-length": "728", "date": "Wed, 30 Apr 2025 06:44:13 GMT"} })
111322025-04-30T06:44:14.117ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0
111332025-04-30T06:44:14.121ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "692fcf93-5689-487d-b80e-aea01d373924", "content-length": "723", "date": "Wed, 30 Apr 2025 06:44:13 GMT"} })
111342025-04-30T06:44:14.121ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0/component/sp/caboose?firmware_slot=0
111352025-04-30T06:44:14.126ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "88b3097a-7519-4d9c-ad08-b4f105c8c289", "content-length": "109", "date": "Wed, 30 Apr 2025 06:44:13 GMT"} })
111362025-04-30T06:44:14.126ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0/component/sp/caboose?firmware_slot=1
111372025-04-30T06:44:14.128ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "7ffa4b3a-44bd-496d-8045-9772fc568fa3", "content-length": "109", "date": "Wed, 30 Apr 2025 06:44:13 GMT"} })
111382025-04-30T06:44:14.128ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0/component/rot/caboose?firmware_slot=0
111392025-04-30T06:44:14.129ZTRACtest_utilization (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory
111402025-04-30T06:44:14.129ZTRACtest_utilization (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
111412025-04-30T06:44:14.129ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
111422025-04-30T06:44:14.130ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
111432025-04-30T06:44:14.131ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
111442025-04-30T06:44:14.134ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
111452025-04-30T06:44:14.142ZDEBGtest_utilization (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory result = Ok(())
111462025-04-30T06:44:14.142ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
111472025-04-30T06:44:14.142ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
111482025-04-30T06:44:14.143ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
111492025-04-30T06:44:14.145ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "6f314697-17d6-468a-9abe-a8a41b7e220c", "content-length": "169", "date": "Wed, 30 Apr 2025 06:44:13 GMT"} })
111502025-04-30T06:44:14.145ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0/component/rot/caboose?firmware_slot=1
111512025-04-30T06:44:14.146ZDEBGtest_utilization (collection-task): sent timer-based collection request to the collection task collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = ac55d8d9-a298-4534-898b-beee1c9e4e52
111522025-04-30T06:44:14.146ZDEBGtest_utilization (collection-task): collecting from producer collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = ac55d8d9-a298-4534-898b-beee1c9e4e52
111532025-04-30T06:44:14.147ZTRACtest_utilization (dropshot): incoming request local_addr = 127.0.0.1:57688 method = GET remote_addr = 127.0.0.1:55243 req_id = 71aa3384-03b1-47a3-9aae-b8f2f1c1c087 uri = /ac55d8d9-a298-4534-898b-beee1c9e4e52
111542025-04-30T06:44:14.161ZINFOtest_utilization (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 13961 local_addr = 127.0.0.1:57688 method = GET remote_addr = 127.0.0.1:55243 req_id = 71aa3384-03b1-47a3-9aae-b8f2f1c1c087 response_code = 200 uri = /ac55d8d9-a298-4534-898b-beee1c9e4e52
111552025-04-30T06:44:14.162ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
111562025-04-30T06:44:14.162ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
111572025-04-30T06:44:14.163ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
111582025-04-30T06:44:14.163ZTRACtest_utilization (collection-task): sent collection request to producer collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = ac55d8d9-a298-4534-898b-beee1c9e4e52
111592025-04-30T06:44:14.179ZDEBGtest_utilization (collection-task): collected results from producer collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 n_results = 5 producer_id = ac55d8d9-a298-4534-898b-beee1c9e4e52
111602025-04-30T06:44:14.179ZTRACtest_utilization (collection-task): forwarded results to main collection loop collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = ac55d8d9-a298-4534-898b-beee1c9e4e52
111612025-04-30T06:44:14.180ZTRACtest_utilization (collection-task): top of inner collection loop, waiting for next request collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = ac55d8d9-a298-4534-898b-beee1c9e4e52
111622025-04-30T06:44:14.180ZDEBGtest_utilization (results-sink): inserting 16 samples into database collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
111632025-04-30T06:44:14.180ZDEBGtest_utilization (results-sink-cluster): inserting 16 samples into database collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
111642025-04-30T06:44:14.180ZDEBGtest_utilization (clickhouse-client): unrolling 16 total samples collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 4d1485f0-110e-4161-9a33-c4610d68be5d
111652025-04-30T06:44:14.194ZDEBGtest_utilization (clickhouse-client): retrieving timeseries schema from database collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 4d1485f0-110e-4161-9a33-c4610d68be5d
111662025-04-30T06:44:14.194ZTRACtest_utilization (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 4d1485f0-110e-4161-9a33-c4610d68be5d sql = SELECT * FROM oximeter.timeseries_schema WHERE timeseries_name NOT IN ('collection_target:cpus_provisioned', 'collection_target:ram_provisioned', 'collection_target:virtual_disk_space_provisioned', 'http_service:request_latency_histogram', 'integration_target:integration_metric') FORMAT Native;
111672025-04-30T06:44:14.199ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "bb144d8d-0edf-4b92-954d-c7efe20b51e2", "content-length": "169", "date": "Wed, 30 Apr 2025 06:44:13 GMT"} })
111682025-04-30T06:44:14.199ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0/component/stage0/caboose?firmware_slot=0
111692025-04-30T06:44:14.200ZTRACtest_utilization (clickhouse-client): no new timeseries schema in database collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 4d1485f0-110e-4161-9a33-c4610d68be5d
111702025-04-30T06:44:14.200ZDEBGtest_utilization (clickhouse-client): new timeseries schema collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 4d1485f0-110e-4161-9a33-c4610d68be5d schema = TimeseriesSchema { timeseries_name: TimeseriesName("virtual_machine:check"), description: TimeseriesDescription { target: "", metric: "" }, field_schema: {FieldSchema { name: "instance_id", field_type: Uuid, source: Target, description: "" }, FieldSchema { name: "nexus_id", field_type: Uuid, source: Target, description: "" }, FieldSchema { name: "project_id", field_type: Uuid, source: Target, description: "" }, FieldSchema { name: "rack_id", field_type: Uuid, source: Target, description: "" }, FieldSchema { name: "reason", field_type: String, source: Metric, description: "" }, FieldSchema { name: "silo_id", field_type: Uuid, source: Target, description: "" }, FieldSchema { name: "sled_agent_id", field_type: Uuid, source: Target, description: "" }, FieldSchema { name: "sled_agent_ip", field_type: IpAddr, source: Target, description: "" }, FieldSchema { name: "sled_agent_port", field_type: U16, source: Target, description: "" }, FieldSchema { name: "state", field_type: String, source: Metric, description: "" }, FieldSchema { name: "vmm_id", field_type: Uuid, source: Target, description: "" }}, datum_type: CumulativeU64, version: 1, authz_scope: Fleet, units: Count, created: 2025-04-30T06:44:14.194777249Z } timeseries_name = virtual_machine:check
111712025-04-30T06:44:14.201ZDEBGtest_utilization (clickhouse-client): inserting 1 new timeseries schema collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 4d1485f0-110e-4161-9a33-c4610d68be5d
111722025-04-30T06:44:14.201ZTRACtest_utilization (clickhouse-client): inserting data collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 4d1485f0-110e-4161-9a33-c4610d68be5d n_columns = 6 n_rows = 1 sql = INSERT INTO oximeter.timeseries_schema FORMAT Native
111732025-04-30T06:44:14.206ZTRACtest_utilization (clickhouse-client): inserting data collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 4d1485f0-110e-4161-9a33-c4610d68be5d n_columns = 4 n_rows = 1 sql = INSERT INTO oximeter.fields_ipaddr FORMAT Native
111742025-04-30T06:44:14.207ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "94b6a1ed-a0ac-4196-89f9-764415d5fad0", "content-length": "178", "date": "Wed, 30 Apr 2025 06:44:13 GMT"} })
111752025-04-30T06:44:14.207ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0/component/stage0/caboose?firmware_slot=1
111762025-04-30T06:44:14.212ZDEBGtest_utilization (clickhouse-client): inserted rows into table collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 4d1485f0-110e-4161-9a33-c4610d68be5d n_rows = 1 table_name = fields_ipaddr
111772025-04-30T06:44:14.213ZTRACtest_utilization (clickhouse-client): inserting data collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 4d1485f0-110e-4161-9a33-c4610d68be5d n_columns = 4 n_rows = 32 sql = INSERT INTO oximeter.fields_string FORMAT Native
111782025-04-30T06:44:14.216ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "56f91aca-5f36-49ff-8590-5af54fa9d6f5", "content-length": "178", "date": "Wed, 30 Apr 2025 06:44:13 GMT"} })
111792025-04-30T06:44:14.216ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0/component/rot/cmpa
111802025-04-30T06:44:14.216ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "03656c50-e945-49b8-86dd-b0e88998f232", "content-length": "702", "date": "Wed, 30 Apr 2025 06:44:13 GMT"} })
111812025-04-30T06:44:14.216ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0/component/rot/cfpa
111822025-04-30T06:44:14.216ZDEBGtest_utilization (clickhouse-client): inserted rows into table collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 4d1485f0-110e-4161-9a33-c4610d68be5d n_rows = 32 table_name = fields_string
111832025-04-30T06:44:14.217ZTRACtest_utilization (clickhouse-client): inserting data collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 4d1485f0-110e-4161-9a33-c4610d68be5d n_columns = 4 n_rows = 15 sql = INSERT INTO oximeter.fields_u16 FORMAT Native
111842025-04-30T06:44:14.217ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "00ad266d-6c4d-4cb2-8585-e239300ea0e8", "content-length": "727", "date": "Wed, 30 Apr 2025 06:44:13 GMT"} })
111852025-04-30T06:44:14.217ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0/component/rot/cfpa
111862025-04-30T06:44:14.220ZDEBGtest_utilization (clickhouse-client): inserted rows into table collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 4d1485f0-110e-4161-9a33-c4610d68be5d n_rows = 15 table_name = fields_u16
111872025-04-30T06:44:14.221ZTRACtest_utilization (clickhouse-client): inserting data collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 4d1485f0-110e-4161-9a33-c4610d68be5d n_columns = 4 n_rows = 21 sql = INSERT INTO oximeter.fields_uuid FORMAT Native
111882025-04-30T06:44:14.221ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "0377d4ac-9560-4c87-bbcc-c627cb376e19", "content-length": "729", "date": "Wed, 30 Apr 2025 06:44:13 GMT"} })
111892025-04-30T06:44:14.221ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0/component/rot/cfpa
111902025-04-30T06:44:14.223ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "cd021ef2-8719-44e9-8954-e90442f31f41", "content-length": "728", "date": "Wed, 30 Apr 2025 06:44:13 GMT"} })
111912025-04-30T06:44:14.223ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1
111922025-04-30T06:44:14.225ZDEBGtest_utilization (clickhouse-client): inserted rows into table collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 4d1485f0-110e-4161-9a33-c4610d68be5d n_rows = 21 table_name = fields_uuid
111932025-04-30T06:44:14.225ZTRACtest_utilization (clickhouse-client): inserting data collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 4d1485f0-110e-4161-9a33-c4610d68be5d n_columns = 5 n_rows = 1 sql = INSERT INTO oximeter.measurements_cumulativeu64 FORMAT Native
111942025-04-30T06:44:14.227ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1", status: 200, headers: {"content-type": "application/json", "x-request-id": "bc731db4-29ab-4cfa-b7f4-fc13ba1f32a5", "content-length": "723", "date": "Wed, 30 Apr 2025 06:44:13 GMT"} })
111952025-04-30T06:44:14.228ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1/component/sp/caboose?firmware_slot=0
111962025-04-30T06:44:14.230ZDEBGtest_utilization (clickhouse-client): inserted rows into table collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 4d1485f0-110e-4161-9a33-c4610d68be5d n_rows = 1 table_name = measurements_cumulativeu64
111972025-04-30T06:44:14.230ZTRACtest_utilization (clickhouse-client): inserting data collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 4d1485f0-110e-4161-9a33-c4610d68be5d n_columns = 19 n_rows = 14 sql = INSERT INTO oximeter.measurements_histogramu64 FORMAT Native
111982025-04-30T06:44:14.231ZDEBGtest_utilization: attempt to fetch latest inventory collection result = Ok("found")
111992025-04-30T06:44:14.235ZDEBGtest_utilization (clickhouse-client): inserted rows into table collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 4d1485f0-110e-4161-9a33-c4610d68be5d n_rows = 14 table_name = measurements_histogramu64
112002025-04-30T06:44:14.235ZTRACtest_utilization (clickhouse-client): inserting data collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 4d1485f0-110e-4161-9a33-c4610d68be5d n_columns = 4 n_rows = 1 sql = INSERT INTO oximeter.measurements_i64 FORMAT Native
112012025-04-30T06:44:14.237ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "1ccdf4d6-b3d2-47d0-aebe-d05c77f14074", "content-length": "109", "date": "Wed, 30 Apr 2025 06:44:13 GMT"} })
112022025-04-30T06:44:14.237ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1/component/sp/caboose?firmware_slot=1
112032025-04-30T06:44:14.241ZDEBGtest_utilization (clickhouse-client): inserted rows into table collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 4d1485f0-110e-4161-9a33-c4610d68be5d n_rows = 1 table_name = measurements_i64
112042025-04-30T06:44:14.241ZTRACtest_utilization (results-sink): successfully inserted samples collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
112052025-04-30T06:44:14.243ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "d41fba2b-8a74-4a01-a587-515d73437150", "content-length": "109", "date": "Wed, 30 Apr 2025 06:44:13 GMT"} })
112062025-04-30T06:44:14.243ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1/component/rot/caboose?firmware_slot=0
112072025-04-30T06:44:14.254ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "f7fae08d-3b12-4b52-ac44-bc0a16078f42", "content-length": "169", "date": "Wed, 30 Apr 2025 06:44:13 GMT"} })
112082025-04-30T06:44:14.254ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1/component/rot/caboose?firmware_slot=1
112092025-04-30T06:44:14.258ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "c777029e-1738-4412-a8dc-cee9d58f7426", "content-length": "169", "date": "Wed, 30 Apr 2025 06:44:13 GMT"} })
112102025-04-30T06:44:14.260ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1/component/stage0/caboose?firmware_slot=0
112112025-04-30T06:44:14.263ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "bcd2223d-4a89-4eee-9c6e-432bb64484f6", "content-length": "178", "date": "Wed, 30 Apr 2025 06:44:13 GMT"} })
112122025-04-30T06:44:14.263ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1/component/stage0/caboose?firmware_slot=1
112132025-04-30T06:44:14.268ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "a4dc98cc-7cfb-41ca-8482-e32a4c421a3d", "content-length": "178", "date": "Wed, 30 Apr 2025 06:44:13 GMT"} })
112142025-04-30T06:44:14.268ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1/component/rot/cmpa
112152025-04-30T06:44:14.270ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "8a741659-e7fb-4ef9-b787-34a5c298335e", "content-length": "702", "date": "Wed, 30 Apr 2025 06:44:13 GMT"} })
112162025-04-30T06:44:14.271ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1/component/rot/cfpa
112172025-04-30T06:44:14.272ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "b94a2ba9-3ccb-473c-b9df-1143a8f5d6b0", "content-length": "727", "date": "Wed, 30 Apr 2025 06:44:13 GMT"} })
112182025-04-30T06:44:14.273ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1/component/rot/cfpa
112192025-04-30T06:44:14.276ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "aa05b434-c1fd-4828-92c2-fdfe28a46ca2", "content-length": "729", "date": "Wed, 30 Apr 2025 06:44:13 GMT"} })
112202025-04-30T06:44:14.276ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1/component/rot/cfpa
112212025-04-30T06:44:14.277ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "10e81659-1496-4860-a20b-27cf1620d1af", "content-length": "728", "date": "Wed, 30 Apr 2025 06:44:13 GMT"} })
112222025-04-30T06:44:14.277ZTRACtest_utilization (BackgroundTasks): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet
112232025-04-30T06:44:14.277ZTRACtest_utilization (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
112242025-04-30T06:44:14.277ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
112252025-04-30T06:44:14.277ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
112262025-04-30T06:44:14.278ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
112272025-04-30T06:44:14.281ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
112282025-04-30T06:44:14.289ZDEBGtest_utilization (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet result = Ok(())
112292025-04-30T06:44:14.289ZTRACtest_utilization (BackgroundTasks): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet
112302025-04-30T06:44:14.289ZTRACtest_utilization (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
112312025-04-30T06:44:14.289ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
112322025-04-30T06:44:14.289ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
112332025-04-30T06:44:14.290ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
112342025-04-30T06:44:14.290ZWARNtest_utilization (results-sink-cluster): failed to insert some results into metric DB collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 error = Failed to check out connection to database: No backends found for this service file = oximeter/collector/src/results_sink.rs:92
112352025-04-30T06:44:14.291ZTRACtest_utilization (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory
112362025-04-30T06:44:14.291ZTRACtest_utilization (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
112372025-04-30T06:44:14.291ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
112382025-04-30T06:44:14.291ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
112392025-04-30T06:44:14.292ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
112402025-04-30T06:44:14.294ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
112412025-04-30T06:44:14.301ZDEBGtest_utilization (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet result = Ok(())
112422025-04-30T06:44:14.302ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
112432025-04-30T06:44:14.302ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
112442025-04-30T06:44:14.303ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
112452025-04-30T06:44:14.304ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
112462025-04-30T06:44:14.313ZDEBGtest_utilization (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory result = Ok(())
112472025-04-30T06:44:14.313ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
112482025-04-30T06:44:14.313ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
112492025-04-30T06:44:14.313ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
112502025-04-30T06:44:14.317ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
112512025-04-30T06:44:14.317ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
112522025-04-30T06:44:14.320ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
112532025-04-30T06:44:14.376ZDEBGtest_utilization (BackgroundTasks): begin collection from Sled Agent background_task = inventory_collection sled_agent_url = http://[::1]:39365
112542025-04-30T06:44:14.376ZDEBGtest_utilization (BackgroundTasks): client request SledAgent = http://[::1]:39365 background_task = inventory_collection body = None method = GET uri = http://[::1]:39365/inventory
112552025-04-30T06:44:14.377ZINFOtest_utilization (dropshot): accepted connection local_addr = [::1]:39365 remote_addr = [::1]:41699 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
112562025-04-30T06:44:14.377ZTRACtest_utilization (results-sink): batch interval expired, but no samples to insert collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
112572025-04-30T06:44:14.377ZTRACtest_utilization (results-sink-cluster): batch interval expired, but no samples to insert collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
112582025-04-30T06:44:14.378ZTRACtest_utilization (dropshot): incoming request local_addr = [::1]:39365 method = GET remote_addr = [::1]:41699 req_id = 1f8bb828-3b04-44d2-b2eb-e514a5aa0267 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /inventory
112592025-04-30T06:44:14.379ZINFOtest_utilization (dropshot): request completed latency_us = 983 local_addr = [::1]:39365 method = GET remote_addr = [::1]:41699 req_id = 1f8bb828-3b04-44d2-b2eb-e514a5aa0267 response_code = 200 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /inventory
112602025-04-30T06:44:14.379ZDEBGtest_utilization (BackgroundTasks): client response SledAgent = http://[::1]:39365 background_task = inventory_collection result = Ok(Response { url: "http://[::1]:39365/inventory", status: 200, headers: {"content-type": "application/json", "x-request-id": "1f8bb828-3b04-44d2-b2eb-e514a5aa0267", "content-length": "3141", "date": "Wed, 30 Apr 2025 06:44:13 GMT"} })
112612025-04-30T06:44:14.383ZDEBGtest_utilization (BackgroundTasks): begin collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
112622025-04-30T06:44:14.383ZDEBGtest_utilization (BackgroundTasks): end collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
112632025-04-30T06:44:14.383ZDEBGtest_utilization (BackgroundTasks): finished collection background_task = inventory_collection
112642025-04-30T06:44:14.385ZTRACtest_utilization (BackgroundTasks): authorize begin action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory
112652025-04-30T06:44:14.385ZTRACtest_utilization (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
112662025-04-30T06:44:14.385ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
112672025-04-30T06:44:14.385ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
112682025-04-30T06:44:14.386ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
112692025-04-30T06:44:14.388ZTRACtest_utilization (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal
112702025-04-30T06:44:14.390ZTRACtest_utilization (dns): buffer buffer = Length: 72 (0x48) bytes\n0000: 5d f8 01 00 00 01 00 00 00 00 00 01 09 5f 64 65 ]............_de\n0010: 6e 64 72 69 74 65 04 5f 74 63 70 0d 63 6f 6e 74 ndrite._tcp.cont\n0020: 72 6f 6c 2d 70 6c 61 6e 65 05 6f 78 69 64 65 08 rol-plane.oxide.\n0030: 69 6e 74 65 72 6e 61 6c 00 00 21 00 01 00 00 29 internal..!....)\n0040: 04 d0 00 00 00 00 00 00 ........ kind = dns peer_addr = [::1]:54982 req_id = d441b146-2854-459c-b09b-c63d7f0a6670
112712025-04-30T06:44:14.390ZDEBGtest_utilization (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 24056,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_dendrite._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_dendrite._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 9,\n 95,\n 100,\n 101,\n 110,\n 100,\n 114,\n 105,\n 116,\n 101,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:54982 req_id = d441b146-2854-459c-b09b-c63d7f0a6670
112722025-04-30T06:44:14.390ZDEBGtest_utilization (store): query key key = _dendrite._tcp
112732025-04-30T06:44:14.390ZDEBGtest_utilization (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
112742025-04-30T06:44:14.391ZDEBGtest_utilization (dns): dns response additional_records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal"), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] kind = dns peer_addr = [::1]:54982 query = LowerQuery { name: LowerName(Name("_dendrite._tcp.control-plane.oxide.internal.")), original: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN } } records = [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 47974, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal") })) }] req_id = d441b146-2854-459c-b09b-c63d7f0a6670
112752025-04-30T06:44:14.391ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
112762025-04-30T06:44:14.397ZDEBGtest_utilization (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory result = Ok(())
112772025-04-30T06:44:14.397ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
112782025-04-30T06:44:14.397ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
112792025-04-30T06:44:14.398ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
112802025-04-30T06:44:14.400ZDEBGtest_utilization (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 47974, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }], valid_until: Instant { tv_sec: 4319, tv_nsec: 199166890 } })
112812025-04-30T06:44:14.400ZTRACtest_utilization (DnsResolver): lookup_service_targets: looking up SRV target
112822025-04-30T06:44:14.406ZTRACtest_utilization (dns): buffer buffer = Length: 108 (0x6c) bytes\n0000: 8b d0 01 00 00 01 00 00 00 00 00 01 2d 64 65 6e ............-den\n0010: 64 72 69 74 65 2d 62 36 64 36 35 33 34 31 2d 31 drite-b6d65341-1\n0020: 36 37 63 2d 34 31 64 66 2d 39 62 35 63 2d 34 31 67c-41df-9b5c-41\n0030: 63 64 65 64 39 39 63 32 32 39 04 68 6f 73 74 0d cded99c229.host.\n0040: 63 6f 6e 74 72 6f 6c 2d 70 6c 61 6e 65 05 6f 78 control-plane.ox\n0050: 69 64 65 08 69 6e 74 65 72 6e 61 6c 00 00 1c 00 ide.internal....\n0060: 01 00 00 29 04 d0 00 00 00 00 00 00 ...)........ kind = dns peer_addr = [::1]:64937 req_id = 86f33937-7bae-46a4-82eb-bbf510b74e48
112832025-04-30T06:44:14.406ZDEBGtest_utilization (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 35792,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n query_type: AAAA,\n query_class: IN,\n },\n },\n original: [\n 45,\n 100,\n 101,\n 110,\n 100,\n 114,\n 105,\n 116,\n 101,\n 45,\n 98,\n 54,\n 100,\n 54,\n 53,\n 51,\n 52,\n 49,\n 45,\n 49,\n 54,\n 55,\n 99,\n 45,\n 52,\n 49,\n 100,\n 102,\n 45,\n 57,\n 98,\n 53,\n 99,\n 45,\n 52,\n 49,\n 99,\n 100,\n 101,\n 100,\n 57,\n 57,\n 99,\n 50,\n 50,\n 57,\n 4,\n 104,\n 111,\n 115,\n 116,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 28,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:64937 req_id = 86f33937-7bae-46a4-82eb-bbf510b74e48
112842025-04-30T06:44:14.407ZDEBGtest_utilization (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
112852025-04-30T06:44:14.407ZDEBGtest_utilization (dns): dns response additional_records = [] kind = dns peer_addr = [::1]:64937 query = LowerQuery { name: LowerName(Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.")), original: Query { name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), query_type: AAAA, query_class: IN } } records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] req_id = 86f33937-7bae-46a4-82eb-bbf510b74e48
112862025-04-30T06:44:14.408ZINFOtest_utilization (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_v4_garbage_collector
112872025-04-30T06:44:14.516ZINFOtest_utilization (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_v4_garbage_collector zone_address = ::1
112882025-04-30T06:44:14.516ZDEBGtest_utilization (MgsClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[::1]:12225/local/switch-id
112892025-04-30T06:44:14.518ZDEBGtest_utilization (MgsClient): client response background_task = nat_v4_garbage_collector result = Err(reqwest::Error { kind: Request, url: "http://[::1]:12225/local/switch-id", source: hyper_util::client::legacy::Error(Connect, ConnectError("tcp connect error", Os { code: 146, kind: ConnectionRefused, message: "Connection refused" })) })
112902025-04-30T06:44:14.518ZWARNtest_utilization (BackgroundTasks): failed to identify switch slot for dendrite background_task = nat_v4_garbage_collector reason = Communication Error: error sending request for url (http://[::1]:12225/local/switch-id): client error (Connect): tcp connect error: Connection refused (os error 146): Connection refused (os error 146) zone_address = ::1
112912025-04-30T06:44:14.520ZWARNtest_utilization (BackgroundTasks): Failed to map switch zone addr: Communication Error: error sending request for url (http://[::1]:12225/local/switch-id), retrying background_task = nat_v4_garbage_collector
112922025-04-30T06:44:14.526ZTRACtest_utilization (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal
112932025-04-30T06:44:14.527ZTRACtest_utilization (dns): buffer buffer = Length: 72 (0x48) bytes\n0000: 87 b2 01 00 00 01 00 00 00 00 00 01 09 5f 64 65 ............._de\n0010: 6e 64 72 69 74 65 04 5f 74 63 70 0d 63 6f 6e 74 ndrite._tcp.cont\n0020: 72 6f 6c 2d 70 6c 61 6e 65 05 6f 78 69 64 65 08 rol-plane.oxide.\n0030: 69 6e 74 65 72 6e 61 6c 00 00 21 00 01 00 00 29 internal..!....)\n0040: 04 d0 00 00 00 00 00 00 ........ kind = dns peer_addr = [::1]:53207 req_id = 0ea59c80-373b-433b-beed-83ccb05e337d
112942025-04-30T06:44:14.527ZDEBGtest_utilization (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 34738,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_dendrite._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_dendrite._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 9,\n 95,\n 100,\n 101,\n 110,\n 100,\n 114,\n 105,\n 116,\n 101,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:53207 req_id = 0ea59c80-373b-433b-beed-83ccb05e337d
112952025-04-30T06:44:14.527ZDEBGtest_utilization (store): query key key = _dendrite._tcp
112962025-04-30T06:44:14.528ZDEBGtest_utilization (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
112972025-04-30T06:44:14.528ZDEBGtest_utilization (dns): dns response additional_records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal"), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] kind = dns peer_addr = [::1]:53207 query = LowerQuery { name: LowerName(Name("_dendrite._tcp.control-plane.oxide.internal.")), original: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN } } records = [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 47974, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal") })) }] req_id = 0ea59c80-373b-433b-beed-83ccb05e337d
112982025-04-30T06:44:14.530ZDEBGtest_utilization (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 47974, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }], valid_until: Instant { tv_sec: 4319, tv_nsec: 329620189 } })
112992025-04-30T06:44:14.530ZTRACtest_utilization (DnsResolver): lookup_service_targets: looking up SRV target
113002025-04-30T06:44:14.531ZTRACtest_utilization (dns): buffer buffer = Length: 108 (0x6c) bytes\n0000: 2a 28 01 00 00 01 00 00 00 00 00 01 2d 64 65 6e *(..........-den\n0010: 64 72 69 74 65 2d 62 36 64 36 35 33 34 31 2d 31 drite-b6d65341-1\n0020: 36 37 63 2d 34 31 64 66 2d 39 62 35 63 2d 34 31 67c-41df-9b5c-41\n0030: 63 64 65 64 39 39 63 32 32 39 04 68 6f 73 74 0d cded99c229.host.\n0040: 63 6f 6e 74 72 6f 6c 2d 70 6c 61 6e 65 05 6f 78 control-plane.ox\n0050: 69 64 65 08 69 6e 74 65 72 6e 61 6c 00 00 1c 00 ide.internal....\n0060: 01 00 00 29 04 d0 00 00 00 00 00 00 ...)........ kind = dns peer_addr = [::1]:62313 req_id = 5245b6bb-862b-4ac3-bbb1-e958595794c1
113012025-04-30T06:44:14.532ZDEBGtest_utilization (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 10792,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n query_type: AAAA,\n query_class: IN,\n },\n },\n original: [\n 45,\n 100,\n 101,\n 110,\n 100,\n 114,\n 105,\n 116,\n 101,\n 45,\n 98,\n 54,\n 100,\n 54,\n 53,\n 51,\n 52,\n 49,\n 45,\n 49,\n 54,\n 55,\n 99,\n 45,\n 52,\n 49,\n 100,\n 102,\n 45,\n 57,\n 98,\n 53,\n 99,\n 45,\n 52,\n 49,\n 99,\n 100,\n 101,\n 100,\n 57,\n 57,\n 99,\n 50,\n 50,\n 57,\n 4,\n 104,\n 111,\n 115,\n 116,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 28,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:62313 req_id = 5245b6bb-862b-4ac3-bbb1-e958595794c1
113022025-04-30T06:44:14.532ZDEBGtest_utilization (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
113032025-04-30T06:44:14.533ZDEBGtest_utilization (dns): dns response additional_records = [] kind = dns peer_addr = [::1]:62313 query = LowerQuery { name: LowerName(Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.")), original: Query { name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), query_type: AAAA, query_class: IN } } records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] req_id = 5245b6bb-862b-4ac3-bbb1-e958595794c1
113042025-04-30T06:44:14.535ZINFOtest_utilization (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager
113052025-04-30T06:44:14.606ZINFOtest_utilization (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager zone_address = ::1
113062025-04-30T06:44:14.606ZDEBGtest_utilization (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[::1]:12225/local/switch-id
113072025-04-30T06:44:14.606ZDEBGtest_utilization (MgsClient): client response background_task = bfd_manager result = Err(reqwest::Error { kind: Request, url: "http://[::1]:12225/local/switch-id", source: hyper_util::client::legacy::Error(Connect, ConnectError("tcp connect error", Os { code: 146, kind: ConnectionRefused, message: "Connection refused" })) })
113082025-04-30T06:44:14.606ZWARNtest_utilization (BackgroundTasks): failed to identify switch slot for dendrite background_task = bfd_manager reason = Communication Error: error sending request for url (http://[::1]:12225/local/switch-id): client error (Connect): tcp connect error: Connection refused (os error 146): Connection refused (os error 146) zone_address = ::1
113092025-04-30T06:44:14.608ZWARNtest_utilization (BackgroundTasks): Failed to map switch zone addr: Communication Error: error sending request for url (http://[::1]:12225/local/switch-id), retrying background_task = bfd_manager
113102025-04-30T06:44:14.632ZDEBGtest_utilization: attempt to fetch latest inventory collection result = Ok("found")
113112025-04-30T06:44:14.639ZWARNtest_utilization (datastore): Non-retryable transaction failure err = DatabaseError(UniqueViolation, "duplicate key value violates unique constraint \\"inv_caboose_pkey\\"") transaction = inventory_insert_collection
113122025-04-30T06:44:14.641ZWARNtest_utilization (BackgroundTasks): inventory collection failed background_task = inventory_collection error = failed to collect inventory: saving inventory to database: Internal Error: unexpected database error: duplicate key value violates unique constraint "inv_caboose_pkey"
113132025-04-30T06:44:14.641ZDEBGtest_utilization (BackgroundTasks): activation complete background_task = inventory_collection elapsed = 750.953346ms iteration = 26 status = {"error":"failed to collect inventory: saving inventory to database: Internal Error: unexpected database error: duplicate key value violates unique constraint \\"inv_caboose_pkey\\""}
113142025-04-30T06:44:14.641ZDEBGtest_utilization (BackgroundTasks): activating background_task = inventory_collection iteration = 27 reason = Signaled
113152025-04-30T06:44:14.641ZTRACtest_utilization (BackgroundTasks): authorize begin action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory
113162025-04-30T06:44:14.641ZTRACtest_utilization (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
113172025-04-30T06:44:14.642ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
113182025-04-30T06:44:14.642ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
113192025-04-30T06:44:14.642ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
113202025-04-30T06:44:14.652ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
113212025-04-30T06:44:14.660ZDEBGtest_utilization (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory result = Ok(())
113222025-04-30T06:44:14.660ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
113232025-04-30T06:44:14.660ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
113242025-04-30T06:44:14.663ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
113252025-04-30T06:44:14.669ZDEBGtest_utilization (BackgroundTasks): inventory_prune_one: nothing eligible for removal (too few) background_task = inventory_collection candidates = [(13d8b8d9-6c6b-47a6-9190-952544edb75b, 0)]
113262025-04-30T06:44:14.671ZTRACtest_utilization (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal
113272025-04-30T06:44:14.682ZTRACtest_utilization (dns): buffer buffer = Length: 67 (0x43) bytes\n0000: 81 3a 01 00 00 01 00 00 00 00 00 01 04 5f 6d 67 .:..........._mg\n0010: 73 04 5f 74 63 70 0d 63 6f 6e 74 72 6f 6c 2d 70 s._tcp.control-p\n0020: 6c 61 6e 65 05 6f 78 69 64 65 08 69 6e 74 65 72 lane.oxide.inter\n0030: 6e 61 6c 00 00 21 00 01 00 00 29 04 d0 00 00 00 nal..!....).....\n0040: 00 00 00 ... kind = dns peer_addr = [::1]:60088 req_id = cc43b77e-dd9c-4615-bb66-f49046c83c76
113282025-04-30T06:44:14.682ZDEBGtest_utilization (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 33082,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_mgs._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_mgs._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 4,\n 95,\n 109,\n 103,\n 115,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:60088 req_id = cc43b77e-dd9c-4615-bb66-f49046c83c76
113292025-04-30T06:44:14.682ZDEBGtest_utilization (store): query key key = _mgs._tcp
113302025-04-30T06:44:14.682ZDEBGtest_utilization (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
113312025-04-30T06:44:14.682ZDEBGtest_utilization (dns): dns response additional_records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal"), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] kind = dns peer_addr = [::1]:60088 query = LowerQuery { name: LowerName(Name("_mgs._tcp.control-plane.oxide.internal.")), original: Query { name: Name("_mgs._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN } } records = [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 48759, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal") })) }] req_id = cc43b77e-dd9c-4615-bb66-f49046c83c76
113322025-04-30T06:44:14.684ZDEBGtest_utilization (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgs._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 48759, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }], valid_until: Instant { tv_sec: 4319, tv_nsec: 483092954 } })
113332025-04-30T06:44:14.684ZTRACtest_utilization (DnsResolver): lookup_service_targets: looking up SRV target
113342025-04-30T06:44:14.684ZTRACtest_utilization (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory
113352025-04-30T06:44:14.684ZTRACtest_utilization (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
113362025-04-30T06:44:14.684ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
113372025-04-30T06:44:14.684ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
113382025-04-30T06:44:14.685ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
113392025-04-30T06:44:14.688ZTRACtest_utilization (dns): buffer buffer = Length: 108 (0x6c) bytes\n0000: 8e 55 01 00 00 01 00 00 00 00 00 01 2d 64 65 6e .U..........-den\n0010: 64 72 69 74 65 2d 62 36 64 36 35 33 34 31 2d 31 drite-b6d65341-1\n0020: 36 37 63 2d 34 31 64 66 2d 39 62 35 63 2d 34 31 67c-41df-9b5c-41\n0030: 63 64 65 64 39 39 63 32 32 39 04 68 6f 73 74 0d cded99c229.host.\n0040: 63 6f 6e 74 72 6f 6c 2d 70 6c 61 6e 65 05 6f 78 control-plane.ox\n0050: 69 64 65 08 69 6e 74 65 72 6e 61 6c 00 00 1c 00 ide.internal....\n0060: 01 00 00 29 04 d0 00 00 00 00 00 00 ...)........ kind = dns peer_addr = [::1]:64545 req_id = b5e1f327-c37d-4038-b554-ee1680e95ddd
113402025-04-30T06:44:14.688ZDEBGtest_utilization (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 36437,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n query_type: AAAA,\n query_class: IN,\n },\n },\n original: [\n 45,\n 100,\n 101,\n 110,\n 100,\n 114,\n 105,\n 116,\n 101,\n 45,\n 98,\n 54,\n 100,\n 54,\n 53,\n 51,\n 52,\n 49,\n 45,\n 49,\n 54,\n 55,\n 99,\n 45,\n 52,\n 49,\n 100,\n 102,\n 45,\n 57,\n 98,\n 53,\n 99,\n 45,\n 52,\n 49,\n 99,\n 100,\n 101,\n 100,\n 57,\n 57,\n 99,\n 50,\n 50,\n 57,\n 4,\n 104,\n 111,\n 115,\n 116,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 28,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:64545 req_id = b5e1f327-c37d-4038-b554-ee1680e95ddd
113412025-04-30T06:44:14.688ZDEBGtest_utilization (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
113422025-04-30T06:44:14.688ZDEBGtest_utilization (dns): dns response additional_records = [] kind = dns peer_addr = [::1]:64545 query = LowerQuery { name: LowerName(Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.")), original: Query { name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), query_type: AAAA, query_class: IN } } records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] req_id = b5e1f327-c37d-4038-b554-ee1680e95ddd
113432025-04-30T06:44:14.746ZTRACtest_utilization (DnsResolver): lookup_all_socket_v6 srv dns_name = _clickhouse-admin-keeper._tcp.control-plane.oxide.internal
113442025-04-30T06:44:14.752ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
113452025-04-30T06:44:14.769ZDEBGtest_utilization (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory result = Ok(())
113462025-04-30T06:44:14.769ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
113472025-04-30T06:44:14.769ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
113482025-04-30T06:44:14.770ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
113492025-04-30T06:44:14.774ZTRACtest_utilization (dns): buffer buffer = Length: 87 (0x57) bytes\n0000: 18 51 01 00 00 01 00 00 00 00 00 01 18 5f 63 6c .Q..........._cl\n0010: 69 63 6b 68 6f 75 73 65 2d 61 64 6d 69 6e 2d 6b ickhouse-admin-k\n0020: 65 65 70 65 72 04 5f 74 63 70 0d 63 6f 6e 74 72 eeper._tcp.contr\n0030: 6f 6c 2d 70 6c 61 6e 65 05 6f 78 69 64 65 08 69 ol-plane.oxide.i\n0040: 6e 74 65 72 6e 61 6c 00 00 21 00 01 00 00 29 04 nternal..!....).\n0050: d0 00 00 00 00 00 00 ....... kind = dns peer_addr = [::1]:63992 req_id = 61a116c8-78cf-45b5-8fff-685ee1e1b50a
113502025-04-30T06:44:14.774ZDEBGtest_utilization (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 6225,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 24,\n 95,\n 99,\n 108,\n 105,\n 99,\n 107,\n 104,\n 111,\n 117,\n 115,\n 101,\n 45,\n 97,\n 100,\n 109,\n 105,\n 110,\n 45,\n 107,\n 101,\n 101,\n 112,\n 101,\n 114,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:63992 req_id = 61a116c8-78cf-45b5-8fff-685ee1e1b50a
113512025-04-30T06:44:14.774ZDEBGtest_utilization (store): query key key = _clickhouse-admin-keeper._tcp
113522025-04-30T06:44:14.774ZERROtest_utilization (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 6225, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 1, }, query: WireQuery { query: LowerQuery { name: LowerName( Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), ), original: Query { name: Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN, }, }, original: [ 24, 95, 99, 108, 105, 99, 107, 104, 111, 117, 115, 101, 45, 97, 100, 109, 105, 110, 45, 107, 101, 101, 112, 101, 114, 4, 95, 116, 99, 112, 13, 99, 111, 110, 116, 114, 111, 108, 45, 112, 108, 97, 110, 101, 5, 111, 120, 105, 100, 101, 8, 105, 110, 116, 101, 114, 110, 97, 108, 0, 0, 33, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: Some( Edns { rcode_high: 0, version: 0, dnssec_ok: false, max_payload: 1232, options: OPT { options: {}, }, }, ), } NXDOMAIN: no records found for name: "_clickhouse-admin-keeper._tcp.control-plane.oxide.internal." kind = dns peer_addr = [::1]:63992 req_id = 61a116c8-78cf-45b5-8fff-685ee1e1b50a
113532025-04-30T06:44:14.776ZDEBGtest_utilization (BackgroundTasks): begin collection background_task = inventory_collection
113542025-04-30T06:44:14.776ZDEBGtest_utilization (BackgroundTasks): begin collection from MGS background_task = inventory_collection mgs_url = http://[::1]:48759
113552025-04-30T06:44:14.776ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/ignition
113562025-04-30T06:44:14.777ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
113572025-04-30T06:44:14.777ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
113582025-04-30T06:44:14.778ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
113592025-04-30T06:44:14.782ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/ignition", status: 200, headers: {"content-type": "application/json", "x-request-id": "5986fea8-0665-451e-b069-dced4a69e515", "content-length": "835", "date": "Wed, 30 Apr 2025 06:44:13 GMT"} })
113602025-04-30T06:44:14.782ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0
113612025-04-30T06:44:14.787ZTRACtest_utilization (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal
113622025-04-30T06:44:14.787ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "13bb1839-b613-4490-97be-1029964f9478", "content-length": "734", "date": "Wed, 30 Apr 2025 06:44:13 GMT"} })
113632025-04-30T06:44:14.788ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0/component/sp/caboose?firmware_slot=0
113642025-04-30T06:44:14.790ZTRACtest_utilization (dns): buffer buffer = Length: 72 (0x48) bytes\n0000: 2e 33 01 00 00 01 00 00 00 00 00 01 09 5f 64 65 .3..........._de\n0010: 6e 64 72 69 74 65 04 5f 74 63 70 0d 63 6f 6e 74 ndrite._tcp.cont\n0020: 72 6f 6c 2d 70 6c 61 6e 65 05 6f 78 69 64 65 08 rol-plane.oxide.\n0030: 69 6e 74 65 72 6e 61 6c 00 00 21 00 01 00 00 29 internal..!....)\n0040: 04 d0 00 00 00 00 00 00 ........ kind = dns peer_addr = [::1]:64097 req_id = 9bbd45de-a10c-4017-afc3-813e6f0f0d01
113652025-04-30T06:44:14.790ZDEBGtest_utilization (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 11827,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_dendrite._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_dendrite._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 9,\n 95,\n 100,\n 101,\n 110,\n 100,\n 114,\n 105,\n 116,\n 101,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:64097 req_id = 9bbd45de-a10c-4017-afc3-813e6f0f0d01
113662025-04-30T06:44:14.790ZDEBGtest_utilization (store): query key key = _dendrite._tcp
113672025-04-30T06:44:14.791ZDEBGtest_utilization (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
113682025-04-30T06:44:14.791ZDEBGtest_utilization (dns): dns response additional_records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal"), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] kind = dns peer_addr = [::1]:64097 query = LowerQuery { name: LowerName(Name("_dendrite._tcp.control-plane.oxide.internal.")), original: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN } } records = [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 47974, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal") })) }] req_id = 9bbd45de-a10c-4017-afc3-813e6f0f0d01
113692025-04-30T06:44:14.792ZDEBGtest_utilization (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 47974, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }], valid_until: Instant { tv_sec: 4319, tv_nsec: 590963890 } })
113702025-04-30T06:44:14.792ZTRACtest_utilization (DnsResolver): lookup_service_targets: looking up SRV target
113712025-04-30T06:44:14.793ZTRACtest_utilization (dns): buffer buffer = Length: 108 (0x6c) bytes\n0000: 12 b6 01 00 00 01 00 00 00 00 00 01 2d 64 65 6e ............-den\n0010: 64 72 69 74 65 2d 62 36 64 36 35 33 34 31 2d 31 drite-b6d65341-1\n0020: 36 37 63 2d 34 31 64 66 2d 39 62 35 63 2d 34 31 67c-41df-9b5c-41\n0030: 63 64 65 64 39 39 63 32 32 39 04 68 6f 73 74 0d cded99c229.host.\n0040: 63 6f 6e 74 72 6f 6c 2d 70 6c 61 6e 65 05 6f 78 control-plane.ox\n0050: 69 64 65 08 69 6e 74 65 72 6e 61 6c 00 00 1c 00 ide.internal....\n0060: 01 00 00 29 04 d0 00 00 00 00 00 00 ...)........ kind = dns peer_addr = [::1]:60803 req_id = ca863f93-3438-45fb-9527-a3227ac40223
113722025-04-30T06:44:14.793ZDEBGtest_utilization (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 4790,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n query_type: AAAA,\n query_class: IN,\n },\n },\n original: [\n 45,\n 100,\n 101,\n 110,\n 100,\n 114,\n 105,\n 116,\n 101,\n 45,\n 98,\n 54,\n 100,\n 54,\n 53,\n 51,\n 52,\n 49,\n 45,\n 49,\n 54,\n 55,\n 99,\n 45,\n 52,\n 49,\n 100,\n 102,\n 45,\n 57,\n 98,\n 53,\n 99,\n 45,\n 52,\n 49,\n 99,\n 100,\n 101,\n 100,\n 57,\n 57,\n 99,\n 50,\n 50,\n 57,\n 4,\n 104,\n 111,\n 115,\n 116,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 28,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:60803 req_id = ca863f93-3438-45fb-9527-a3227ac40223
113732025-04-30T06:44:14.793ZDEBGtest_utilization (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
113742025-04-30T06:44:14.793ZDEBGtest_utilization (dns): dns response additional_records = [] kind = dns peer_addr = [::1]:60803 query = LowerQuery { name: LowerName(Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.")), original: Query { name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), query_type: AAAA, query_class: IN } } records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] req_id = ca863f93-3438-45fb-9527-a3227ac40223
113752025-04-30T06:44:14.796ZINFOtest_utilization (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager rack_id = c19a698f-c6f9-4a17-ae30-20d711b8f7dc
113762025-04-30T06:44:14.875ZINFOtest_utilization (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager rack_id = c19a698f-c6f9-4a17-ae30-20d711b8f7dc zone_address = ::1
113772025-04-30T06:44:14.876ZDEBGtest_utilization (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = c19a698f-c6f9-4a17-ae30-20d711b8f7dc uri = http://[::1]:12225/local/switch-id
113782025-04-30T06:44:14.876ZDEBGtest_utilization (MgsClient): client response background_task = switch_port_config_manager rack_id = c19a698f-c6f9-4a17-ae30-20d711b8f7dc result = Err(reqwest::Error { kind: Request, url: "http://[::1]:12225/local/switch-id", source: hyper_util::client::legacy::Error(Connect, ConnectError("tcp connect error", Os { code: 146, kind: ConnectionRefused, message: "Connection refused" })) })
113792025-04-30T06:44:14.876ZWARNtest_utilization (BackgroundTasks): failed to identify switch slot for dendrite background_task = switch_port_config_manager rack_id = c19a698f-c6f9-4a17-ae30-20d711b8f7dc reason = Communication Error: error sending request for url (http://[::1]:12225/local/switch-id): client error (Connect): tcp connect error: Connection refused (os error 146): Connection refused (os error 146) zone_address = ::1
113802025-04-30T06:44:14.879ZWARNtest_utilization (BackgroundTasks): Failed to map switch zone addr: Communication Error: error sending request for url (http://[::1]:12225/local/switch-id), retrying background_task = switch_port_config_manager rack_id = c19a698f-c6f9-4a17-ae30-20d711b8f7dc
113812025-04-30T06:44:14.881ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "7f909dd7-adcf-4a2d-a270-4e3294804e25", "content-length": "111", "date": "Wed, 30 Apr 2025 06:44:14 GMT"} })
113822025-04-30T06:44:14.881ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0/component/sp/caboose?firmware_slot=1
113832025-04-30T06:44:14.885ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "0a725dd2-3488-4955-a5e5-206d7af4fcf3", "content-length": "111", "date": "Wed, 30 Apr 2025 06:44:14 GMT"} })
113842025-04-30T06:44:14.885ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0/component/rot/caboose?firmware_slot=0
113852025-04-30T06:44:14.888ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "9b182b35-59cf-4619-8058-420c3e479ef8", "content-length": "170", "date": "Wed, 30 Apr 2025 06:44:14 GMT"} })
113862025-04-30T06:44:14.888ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0/component/rot/caboose?firmware_slot=1
113872025-04-30T06:44:14.890ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "231d75fa-cbef-4299-bbc7-f3d6b78781e7", "content-length": "170", "date": "Wed, 30 Apr 2025 06:44:14 GMT"} })
113882025-04-30T06:44:14.890ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0/component/stage0/caboose?firmware_slot=0
113892025-04-30T06:44:14.893ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "0ebf9522-20c9-49e2-89d9-fb01ac73b2e5", "content-length": "179", "date": "Wed, 30 Apr 2025 06:44:14 GMT"} })
113902025-04-30T06:44:14.893ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0/component/stage0/caboose?firmware_slot=1
113912025-04-30T06:44:14.895ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "3ffdf4d1-fd05-43a1-882f-8d7e64bf0124", "content-length": "179", "date": "Wed, 30 Apr 2025 06:44:14 GMT"} })
113922025-04-30T06:44:14.895ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0/component/rot/cmpa
113932025-04-30T06:44:14.897ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "cd726214-4e9b-41e0-95c5-f4b8cca98b5a", "content-length": "702", "date": "Wed, 30 Apr 2025 06:44:14 GMT"} })
113942025-04-30T06:44:14.898ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0/component/rot/cfpa
113952025-04-30T06:44:14.898ZDEBGtest_utilization: attempt to fetch latest inventory collection result = Ok("found")
113962025-04-30T06:44:14.899ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "da6d99ab-f937-48a1-9ad9-fdc9e590a3ce", "content-length": "727", "date": "Wed, 30 Apr 2025 06:44:14 GMT"} })
113972025-04-30T06:44:14.899ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0/component/rot/cfpa
113982025-04-30T06:44:14.913ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "3f95dd04-fef3-473b-bcf0-75e9a8c1d8f5", "content-length": "729", "date": "Wed, 30 Apr 2025 06:44:14 GMT"} })
113992025-04-30T06:44:14.913ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0/component/rot/cfpa
114002025-04-30T06:44:14.915ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "fdbd011a-2e51-4531-84b2-e6a348bcc1a1", "content-length": "728", "date": "Wed, 30 Apr 2025 06:44:14 GMT"} })
114012025-04-30T06:44:14.915ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1
114022025-04-30T06:44:14.916ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1", status: 200, headers: {"content-type": "application/json", "x-request-id": "eec762d0-9a13-47c8-8c04-695acdd8ccca", "content-length": "734", "date": "Wed, 30 Apr 2025 06:44:14 GMT"} })
114032025-04-30T06:44:14.917ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1/component/sp/caboose?firmware_slot=0
114042025-04-30T06:44:14.919ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "efc69c7b-f09c-40ec-8052-dc06acd536fa", "content-length": "111", "date": "Wed, 30 Apr 2025 06:44:14 GMT"} })
114052025-04-30T06:44:14.919ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1/component/sp/caboose?firmware_slot=1
114062025-04-30T06:44:14.923ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "21808a69-c343-40ae-ae91-16c4bb4d265e", "content-length": "111", "date": "Wed, 30 Apr 2025 06:44:14 GMT"} })
114072025-04-30T06:44:14.923ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1/component/rot/caboose?firmware_slot=0
114082025-04-30T06:44:14.929ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "e5011942-3227-45ba-b41d-99e42cd9d0df", "content-length": "170", "date": "Wed, 30 Apr 2025 06:44:14 GMT"} })
114092025-04-30T06:44:14.929ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1/component/rot/caboose?firmware_slot=1
114102025-04-30T06:44:14.932ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "38e4d560-7042-4b00-9506-e27e6007980b", "content-length": "170", "date": "Wed, 30 Apr 2025 06:44:14 GMT"} })
114112025-04-30T06:44:14.933ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1/component/stage0/caboose?firmware_slot=0
114122025-04-30T06:44:14.936ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "609bbf52-a722-4297-a952-67d92bbc6996", "content-length": "179", "date": "Wed, 30 Apr 2025 06:44:14 GMT"} })
114132025-04-30T06:44:14.936ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1/component/stage0/caboose?firmware_slot=1
114142025-04-30T06:44:14.940ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "c3a41ab6-6ec0-486d-8ad8-a98e3419d88d", "content-length": "179", "date": "Wed, 30 Apr 2025 06:44:14 GMT"} })
114152025-04-30T06:44:14.942ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1/component/rot/cmpa
114162025-04-30T06:44:14.942ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "4c24e57a-ac35-4f9e-8661-4b53310306f5", "content-length": "702", "date": "Wed, 30 Apr 2025 06:44:14 GMT"} })
114172025-04-30T06:44:14.943ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1/component/rot/cfpa
114182025-04-30T06:44:14.944ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "2d32bb9d-f445-4b2e-a871-021496754ab3", "content-length": "727", "date": "Wed, 30 Apr 2025 06:44:14 GMT"} })
114192025-04-30T06:44:14.944ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1/component/rot/cfpa
114202025-04-30T06:44:14.945ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "227838e0-36d4-444e-9815-dcfaa49746a8", "content-length": "729", "date": "Wed, 30 Apr 2025 06:44:14 GMT"} })
114212025-04-30T06:44:14.945ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1/component/rot/cfpa
114222025-04-30T06:44:14.946ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "c87f08ee-b65b-421c-a9d6-bbfbc15f8e14", "content-length": "728", "date": "Wed, 30 Apr 2025 06:44:14 GMT"} })
114232025-04-30T06:44:14.946ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0
114242025-04-30T06:44:14.948ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "b37b4b36-d1f4-4b7f-8e98-7bac048d606d", "content-length": "723", "date": "Wed, 30 Apr 2025 06:44:14 GMT"} })
114252025-04-30T06:44:14.948ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0/component/sp/caboose?firmware_slot=0
114262025-04-30T06:44:14.949ZTRACtest_utilization (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory
114272025-04-30T06:44:14.950ZTRACtest_utilization (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
114282025-04-30T06:44:14.950ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
114292025-04-30T06:44:14.950ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
114302025-04-30T06:44:14.953ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
114312025-04-30T06:44:14.954ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "bd0e8ab4-33c8-4269-9194-b373771f631a", "content-length": "109", "date": "Wed, 30 Apr 2025 06:44:14 GMT"} })
114322025-04-30T06:44:14.954ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0/component/sp/caboose?firmware_slot=1
114332025-04-30T06:44:14.956ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
114342025-04-30T06:44:14.962ZDEBGtest_utilization (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory result = Ok(())
114352025-04-30T06:44:14.962ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
114362025-04-30T06:44:14.962ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
114372025-04-30T06:44:14.963ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
114382025-04-30T06:44:14.965ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
114392025-04-30T06:44:14.965ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
114402025-04-30T06:44:14.966ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
114412025-04-30T06:44:14.966ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "e5679845-1893-40ce-a3a3-295d152c4129", "content-length": "109", "date": "Wed, 30 Apr 2025 06:44:14 GMT"} })
114422025-04-30T06:44:14.966ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0/component/rot/caboose?firmware_slot=0
114432025-04-30T06:44:14.969ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "fdf035d4-f3a7-4f17-a392-60d39395f166", "content-length": "169", "date": "Wed, 30 Apr 2025 06:44:14 GMT"} })
114442025-04-30T06:44:14.969ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0/component/rot/caboose?firmware_slot=1
114452025-04-30T06:44:14.971ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "84d415af-7f56-4794-aa64-f0cf82e78461", "content-length": "169", "date": "Wed, 30 Apr 2025 06:44:14 GMT"} })
114462025-04-30T06:44:14.971ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0/component/stage0/caboose?firmware_slot=0
114472025-04-30T06:44:14.975ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "db2b1500-d665-4544-9efb-d9f34f8d1116", "content-length": "178", "date": "Wed, 30 Apr 2025 06:44:14 GMT"} })
114482025-04-30T06:44:14.975ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0/component/stage0/caboose?firmware_slot=1
114492025-04-30T06:44:14.989ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "d33a635a-3331-4605-b949-523e9cdc1544", "content-length": "178", "date": "Wed, 30 Apr 2025 06:44:14 GMT"} })
114502025-04-30T06:44:14.989ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0/component/rot/cmpa
114512025-04-30T06:44:14.990ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "5d473cee-0ebb-4910-91ba-592ab4a424af", "content-length": "702", "date": "Wed, 30 Apr 2025 06:44:14 GMT"} })
114522025-04-30T06:44:14.990ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0/component/rot/cfpa
114532025-04-30T06:44:14.993ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "7e8128fd-1543-4e6a-8c09-fd94e5dadac5", "content-length": "727", "date": "Wed, 30 Apr 2025 06:44:14 GMT"} })
114542025-04-30T06:44:14.993ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0/component/rot/cfpa
114552025-04-30T06:44:14.995ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "3d3d8245-d389-4dcc-92d5-4243bc2696f9", "content-length": "729", "date": "Wed, 30 Apr 2025 06:44:14 GMT"} })
114562025-04-30T06:44:14.995ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0/component/rot/cfpa
114572025-04-30T06:44:14.996ZDEBGtest_utilization (collection-task): sent timer-based collection request to the collection task collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
114582025-04-30T06:44:14.997ZDEBGtest_utilization (collection-task): collecting from producer collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
114592025-04-30T06:44:14.999ZTRACtest_utilization (collection-task): sent collection request to producer collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
114602025-04-30T06:44:14.999ZDEBGtest_utilization (collection-task): collected results from producer collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 n_results = 1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
114612025-04-30T06:44:14.999ZTRACtest_utilization (collection-task): forwarded results to main collection loop collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
114622025-04-30T06:44:14.999ZTRACtest_utilization (collection-task): top of inner collection loop, waiting for next request collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
114632025-04-30T06:44:15.000ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "fd925456-d6e9-4d2f-9e12-26ce6f1dd04d", "content-length": "728", "date": "Wed, 30 Apr 2025 06:44:14 GMT"} })
114642025-04-30T06:44:15.000ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1
114652025-04-30T06:44:15.002ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1", status: 200, headers: {"content-type": "application/json", "x-request-id": "8b3759d9-7e15-416f-9d4a-5fcf6fe09b28", "content-length": "723", "date": "Wed, 30 Apr 2025 06:44:14 GMT"} })
114662025-04-30T06:44:15.002ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1/component/sp/caboose?firmware_slot=0
114672025-04-30T06:44:15.009ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "cedcdad1-71bc-4df7-8a11-2099973245be", "content-length": "109", "date": "Wed, 30 Apr 2025 06:44:14 GMT"} })
114682025-04-30T06:44:15.010ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1/component/sp/caboose?firmware_slot=1
114692025-04-30T06:44:15.011ZDEBGtest_utilization: attempt to fetch latest inventory collection result = Ok("found")
114702025-04-30T06:44:15.015ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "e30e9616-eb7f-43f0-ac68-569d7ef85122", "content-length": "109", "date": "Wed, 30 Apr 2025 06:44:14 GMT"} })
114712025-04-30T06:44:15.015ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1/component/rot/caboose?firmware_slot=0
114722025-04-30T06:44:15.017ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "1cbc346f-a1ab-4a8e-9a78-3e50eacd3891", "content-length": "169", "date": "Wed, 30 Apr 2025 06:44:14 GMT"} })
114732025-04-30T06:44:15.018ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1/component/rot/caboose?firmware_slot=1
114742025-04-30T06:44:15.021ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "4d0fa431-8029-4f36-9453-dde37f320cfd", "content-length": "169", "date": "Wed, 30 Apr 2025 06:44:14 GMT"} })
114752025-04-30T06:44:15.021ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1/component/stage0/caboose?firmware_slot=0
114762025-04-30T06:44:15.025ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "2b93f1ef-53a3-43a7-9162-593d1054a1bc", "content-length": "178", "date": "Wed, 30 Apr 2025 06:44:14 GMT"} })
114772025-04-30T06:44:15.025ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1/component/stage0/caboose?firmware_slot=1
114782025-04-30T06:44:15.028ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "e5f3aef8-8cd1-4b6e-add2-47ea8e66e7b2", "content-length": "178", "date": "Wed, 30 Apr 2025 06:44:14 GMT"} })
114792025-04-30T06:44:15.028ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1/component/rot/cmpa
114802025-04-30T06:44:15.029ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "117a8bbf-9c77-4137-bf95-aa92e7995160", "content-length": "702", "date": "Wed, 30 Apr 2025 06:44:14 GMT"} })
114812025-04-30T06:44:15.030ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1/component/rot/cfpa
114822025-04-30T06:44:15.032ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "021c96a7-3949-46f5-95cf-a583663d14c3", "content-length": "727", "date": "Wed, 30 Apr 2025 06:44:14 GMT"} })
114832025-04-30T06:44:15.033ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1/component/rot/cfpa
114842025-04-30T06:44:15.034ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "2cae7542-019b-4a67-a270-2d767d5287dc", "content-length": "729", "date": "Wed, 30 Apr 2025 06:44:14 GMT"} })
114852025-04-30T06:44:15.034ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1/component/rot/cfpa
114862025-04-30T06:44:15.035ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "ae8c3d4b-034d-4692-8fdf-209725993f8a", "content-length": "728", "date": "Wed, 30 Apr 2025 06:44:14 GMT"} })
114872025-04-30T06:44:15.035ZTRACtest_utilization (BackgroundTasks): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet
114882025-04-30T06:44:15.035ZTRACtest_utilization (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
114892025-04-30T06:44:15.036ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
114902025-04-30T06:44:15.036ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
114912025-04-30T06:44:15.038ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
114922025-04-30T06:44:15.042ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
114932025-04-30T06:44:15.048ZDEBGtest_utilization (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet result = Ok(())
114942025-04-30T06:44:15.049ZTRACtest_utilization (BackgroundTasks): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet
114952025-04-30T06:44:15.049ZTRACtest_utilization (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
114962025-04-30T06:44:15.049ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
114972025-04-30T06:44:15.049ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
114982025-04-30T06:44:15.050ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
114992025-04-30T06:44:15.054ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
115002025-04-30T06:44:15.062ZDEBGtest_utilization (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet result = Ok(())
115012025-04-30T06:44:15.062ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
115022025-04-30T06:44:15.062ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
115032025-04-30T06:44:15.063ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
115042025-04-30T06:44:15.064ZTRACtest_utilization (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory
115052025-04-30T06:44:15.064ZTRACtest_utilization (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
115062025-04-30T06:44:15.064ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
115072025-04-30T06:44:15.064ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
115082025-04-30T06:44:15.065ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
115092025-04-30T06:44:15.135ZDEBGtest_utilization (BackgroundTasks): begin collection from Sled Agent background_task = inventory_collection sled_agent_url = http://[::1]:39365
115102025-04-30T06:44:15.135ZDEBGtest_utilization (BackgroundTasks): client request SledAgent = http://[::1]:39365 background_task = inventory_collection body = None method = GET uri = http://[::1]:39365/inventory
115112025-04-30T06:44:15.136ZINFOtest_utilization (dropshot): accepted connection local_addr = [::1]:39365 remote_addr = [::1]:49138 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
115122025-04-30T06:44:15.138ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
115132025-04-30T06:44:15.144ZDEBGtest_utilization (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory result = Ok(())
115142025-04-30T06:44:15.144ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
115152025-04-30T06:44:15.144ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
115162025-04-30T06:44:15.145ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
115172025-04-30T06:44:15.145ZTRACtest_utilization (dropshot): incoming request local_addr = [::1]:39365 method = GET remote_addr = [::1]:49138 req_id = a828e961-43ff-4bc6-a94f-c7a4f78a5d25 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /inventory
115182025-04-30T06:44:15.147ZINFOtest_utilization (dropshot): request completed latency_us = 2057 local_addr = [::1]:39365 method = GET remote_addr = [::1]:49138 req_id = a828e961-43ff-4bc6-a94f-c7a4f78a5d25 response_code = 200 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /inventory
115192025-04-30T06:44:15.148ZDEBGtest_utilization (BackgroundTasks): client response SledAgent = http://[::1]:39365 background_task = inventory_collection result = Ok(Response { url: "http://[::1]:39365/inventory", status: 200, headers: {"content-type": "application/json", "x-request-id": "a828e961-43ff-4bc6-a94f-c7a4f78a5d25", "content-length": "3141", "date": "Wed, 30 Apr 2025 06:44:14 GMT"} })
115202025-04-30T06:44:15.151ZDEBGtest_utilization (BackgroundTasks): begin collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
115212025-04-30T06:44:15.151ZDEBGtest_utilization (BackgroundTasks): end collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
115222025-04-30T06:44:15.151ZDEBGtest_utilization (BackgroundTasks): finished collection background_task = inventory_collection
115232025-04-30T06:44:15.157ZTRACtest_utilization (BackgroundTasks): authorize begin action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory
115242025-04-30T06:44:15.157ZTRACtest_utilization (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
115252025-04-30T06:44:15.157ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
115262025-04-30T06:44:15.157ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
115272025-04-30T06:44:15.159ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
115282025-04-30T06:44:15.162ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
115292025-04-30T06:44:15.162ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
115302025-04-30T06:44:15.163ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
115312025-04-30T06:44:15.163ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
115322025-04-30T06:44:15.169ZDEBGtest_utilization (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory result = Ok(())
115332025-04-30T06:44:15.169ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
115342025-04-30T06:44:15.169ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
115352025-04-30T06:44:15.169ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
115362025-04-30T06:44:15.205ZWARNtest_utilization (datastore): Non-retryable transaction failure err = DatabaseError(UniqueViolation, "duplicate key value violates unique constraint \\"inv_caboose_pkey\\"") transaction = inventory_insert_collection
115372025-04-30T06:44:15.205ZWARNtest_utilization (BackgroundTasks): inventory collection failed background_task = inventory_collection error = failed to collect inventory: saving inventory to database: Internal Error: unexpected database error: duplicate key value violates unique constraint "inv_caboose_pkey"
115382025-04-30T06:44:15.206ZDEBGtest_utilization (BackgroundTasks): activation complete background_task = inventory_collection elapsed = 565.171838ms iteration = 27 status = {"error":"failed to collect inventory: saving inventory to database: Internal Error: unexpected database error: duplicate key value violates unique constraint \\"inv_caboose_pkey\\""}
115392025-04-30T06:44:15.206ZDEBGtest_utilization (BackgroundTasks): activating background_task = inventory_collection iteration = 28 reason = Signaled
115402025-04-30T06:44:15.206ZTRACtest_utilization (BackgroundTasks): authorize begin action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory
115412025-04-30T06:44:15.207ZTRACtest_utilization (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
115422025-04-30T06:44:15.207ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
115432025-04-30T06:44:15.207ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
115442025-04-30T06:44:15.207ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
115452025-04-30T06:44:15.211ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
115462025-04-30T06:44:15.216ZDEBGtest_utilization (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory result = Ok(())
115472025-04-30T06:44:15.216ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
115482025-04-30T06:44:15.216ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
115492025-04-30T06:44:15.216ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
115502025-04-30T06:44:15.220ZDEBGtest_utilization: attempt to fetch latest inventory collection result = Ok("found")
115512025-04-30T06:44:15.220ZDEBGtest_utilization (BackgroundTasks): inventory_prune_one: nothing eligible for removal (too few) background_task = inventory_collection candidates = [(13d8b8d9-6c6b-47a6-9190-952544edb75b, 0)]
115522025-04-30T06:44:15.221ZTRACtest_utilization (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal
115532025-04-30T06:44:15.222ZTRACtest_utilization (dns): buffer buffer = Length: 67 (0x43) bytes\n0000: 33 4f 01 00 00 01 00 00 00 00 00 01 04 5f 6d 67 3O..........._mg\n0010: 73 04 5f 74 63 70 0d 63 6f 6e 74 72 6f 6c 2d 70 s._tcp.control-p\n0020: 6c 61 6e 65 05 6f 78 69 64 65 08 69 6e 74 65 72 lane.oxide.inter\n0030: 6e 61 6c 00 00 21 00 01 00 00 29 04 d0 00 00 00 nal..!....).....\n0040: 00 00 00 ... kind = dns peer_addr = [::1]:56343 req_id = 27f6d6c8-c215-4e16-bd8d-8b0cc15413b3
115542025-04-30T06:44:15.222ZDEBGtest_utilization (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 13135,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_mgs._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_mgs._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 4,\n 95,\n 109,\n 103,\n 115,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:56343 req_id = 27f6d6c8-c215-4e16-bd8d-8b0cc15413b3
115552025-04-30T06:44:15.222ZDEBGtest_utilization (store): query key key = _mgs._tcp
115562025-04-30T06:44:15.222ZDEBGtest_utilization (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
115572025-04-30T06:44:15.223ZDEBGtest_utilization (dns): dns response additional_records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal"), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] kind = dns peer_addr = [::1]:56343 query = LowerQuery { name: LowerName(Name("_mgs._tcp.control-plane.oxide.internal.")), original: Query { name: Name("_mgs._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN } } records = [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 48759, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal") })) }] req_id = 27f6d6c8-c215-4e16-bd8d-8b0cc15413b3
115582025-04-30T06:44:15.223ZDEBGtest_utilization (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgs._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 48759, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }], valid_until: Instant { tv_sec: 4320, tv_nsec: 22275851 } })
115592025-04-30T06:44:15.223ZTRACtest_utilization (DnsResolver): lookup_service_targets: looking up SRV target
115602025-04-30T06:44:15.223ZTRACtest_utilization (dns): buffer buffer = Length: 108 (0x6c) bytes\n0000: 48 18 01 00 00 01 00 00 00 00 00 01 2d 64 65 6e H...........-den\n0010: 64 72 69 74 65 2d 62 36 64 36 35 33 34 31 2d 31 drite-b6d65341-1\n0020: 36 37 63 2d 34 31 64 66 2d 39 62 35 63 2d 34 31 67c-41df-9b5c-41\n0030: 63 64 65 64 39 39 63 32 32 39 04 68 6f 73 74 0d cded99c229.host.\n0040: 63 6f 6e 74 72 6f 6c 2d 70 6c 61 6e 65 05 6f 78 control-plane.ox\n0050: 69 64 65 08 69 6e 74 65 72 6e 61 6c 00 00 1c 00 ide.internal....\n0060: 01 00 00 29 04 d0 00 00 00 00 00 00 ...)........ kind = dns peer_addr = [::1]:49637 req_id = 263fb1ae-f430-4a95-9652-3769a58ec366
115612025-04-30T06:44:15.223ZDEBGtest_utilization (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 18456,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n query_type: AAAA,\n query_class: IN,\n },\n },\n original: [\n 45,\n 100,\n 101,\n 110,\n 100,\n 114,\n 105,\n 116,\n 101,\n 45,\n 98,\n 54,\n 100,\n 54,\n 53,\n 51,\n 52,\n 49,\n 45,\n 49,\n 54,\n 55,\n 99,\n 45,\n 52,\n 49,\n 100,\n 102,\n 45,\n 57,\n 98,\n 53,\n 99,\n 45,\n 52,\n 49,\n 99,\n 100,\n 101,\n 100,\n 57,\n 57,\n 99,\n 50,\n 50,\n 57,\n 4,\n 104,\n 111,\n 115,\n 116,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 28,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:49637 req_id = 263fb1ae-f430-4a95-9652-3769a58ec366
115622025-04-30T06:44:15.223ZDEBGtest_utilization (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
115632025-04-30T06:44:15.224ZDEBGtest_utilization (dns): dns response additional_records = [] kind = dns peer_addr = [::1]:49637 query = LowerQuery { name: LowerName(Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.")), original: Query { name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), query_type: AAAA, query_class: IN } } records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] req_id = 263fb1ae-f430-4a95-9652-3769a58ec366
115642025-04-30T06:44:15.287ZTRACtest_utilization (DnsResolver): lookup_all_socket_v6 srv dns_name = _clickhouse-admin-keeper._tcp.control-plane.oxide.internal
115652025-04-30T06:44:15.289ZTRACtest_utilization (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory
115662025-04-30T06:44:15.290ZTRACtest_utilization (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
115672025-04-30T06:44:15.290ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
115682025-04-30T06:44:15.290ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
115692025-04-30T06:44:15.291ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
115702025-04-30T06:44:15.292ZTRACtest_utilization (dns): buffer buffer = Length: 87 (0x57) bytes\n0000: 72 93 01 00 00 01 00 00 00 00 00 01 18 5f 63 6c r............_cl\n0010: 69 63 6b 68 6f 75 73 65 2d 61 64 6d 69 6e 2d 6b ickhouse-admin-k\n0020: 65 65 70 65 72 04 5f 74 63 70 0d 63 6f 6e 74 72 eeper._tcp.contr\n0030: 6f 6c 2d 70 6c 61 6e 65 05 6f 78 69 64 65 08 69 ol-plane.oxide.i\n0040: 6e 74 65 72 6e 61 6c 00 00 21 00 01 00 00 29 04 nternal..!....).\n0050: d0 00 00 00 00 00 00 ....... kind = dns peer_addr = [::1]:57294 req_id = 910daed8-6174-4461-8f11-a04f0baef37b
115712025-04-30T06:44:15.292ZDEBGtest_utilization (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 29331,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 24,\n 95,\n 99,\n 108,\n 105,\n 99,\n 107,\n 104,\n 111,\n 117,\n 115,\n 101,\n 45,\n 97,\n 100,\n 109,\n 105,\n 110,\n 45,\n 107,\n 101,\n 101,\n 112,\n 101,\n 114,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:57294 req_id = 910daed8-6174-4461-8f11-a04f0baef37b
115722025-04-30T06:44:15.292ZDEBGtest_utilization (store): query key key = _clickhouse-admin-keeper._tcp
115732025-04-30T06:44:15.293ZERROtest_utilization (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 29331, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 1, }, query: WireQuery { query: LowerQuery { name: LowerName( Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), ), original: Query { name: Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN, }, }, original: [ 24, 95, 99, 108, 105, 99, 107, 104, 111, 117, 115, 101, 45, 97, 100, 109, 105, 110, 45, 107, 101, 101, 112, 101, 114, 4, 95, 116, 99, 112, 13, 99, 111, 110, 116, 114, 111, 108, 45, 112, 108, 97, 110, 101, 5, 111, 120, 105, 100, 101, 8, 105, 110, 116, 101, 114, 110, 97, 108, 0, 0, 33, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: Some( Edns { rcode_high: 0, version: 0, dnssec_ok: false, max_payload: 1232, options: OPT { options: {}, }, }, ), } NXDOMAIN: no records found for name: "_clickhouse-admin-keeper._tcp.control-plane.oxide.internal." kind = dns peer_addr = [::1]:57294 req_id = 910daed8-6174-4461-8f11-a04f0baef37b
115742025-04-30T06:44:15.294ZDEBGtest_utilization (BackgroundTasks): begin collection background_task = inventory_collection
115752025-04-30T06:44:15.294ZDEBGtest_utilization (BackgroundTasks): begin collection from MGS background_task = inventory_collection mgs_url = http://[::1]:48759
115762025-04-30T06:44:15.294ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/ignition
115772025-04-30T06:44:15.296ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
115782025-04-30T06:44:15.305ZDEBGtest_utilization (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory result = Ok(())
115792025-04-30T06:44:15.305ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
115802025-04-30T06:44:15.305ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
115812025-04-30T06:44:15.307ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
115822025-04-30T06:44:15.309ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/ignition", status: 200, headers: {"content-type": "application/json", "x-request-id": "b6734cf0-8095-4985-9143-571a9a325341", "content-length": "835", "date": "Wed, 30 Apr 2025 06:44:14 GMT"} })
115832025-04-30T06:44:15.309ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0
115842025-04-30T06:44:15.310ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
115852025-04-30T06:44:15.310ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
115862025-04-30T06:44:15.310ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
115872025-04-30T06:44:15.311ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "757439a4-3fa9-4d07-ac3a-b6f78c018740", "content-length": "734", "date": "Wed, 30 Apr 2025 06:44:14 GMT"} })
115882025-04-30T06:44:15.312ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0/component/sp/caboose?firmware_slot=0
115892025-04-30T06:44:15.314ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "004ef2d7-91fa-44fd-ad9c-b6cca1b1c924", "content-length": "111", "date": "Wed, 30 Apr 2025 06:44:14 GMT"} })
115902025-04-30T06:44:15.314ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0/component/sp/caboose?firmware_slot=1
115912025-04-30T06:44:15.317ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "fa44a28e-4077-4ae7-af11-5976674401e6", "content-length": "111", "date": "Wed, 30 Apr 2025 06:44:14 GMT"} })
115922025-04-30T06:44:15.317ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0/component/rot/caboose?firmware_slot=0
115932025-04-30T06:44:15.319ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "ee6d7d99-d4ff-4ca8-a439-9b2568f91913", "content-length": "170", "date": "Wed, 30 Apr 2025 06:44:14 GMT"} })
115942025-04-30T06:44:15.319ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0/component/rot/caboose?firmware_slot=1
115952025-04-30T06:44:15.325ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "c8dd74d2-0774-4aa7-ad77-21a9cd82c1fc", "content-length": "170", "date": "Wed, 30 Apr 2025 06:44:14 GMT"} })
115962025-04-30T06:44:15.325ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0/component/stage0/caboose?firmware_slot=0
115972025-04-30T06:44:15.333ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "79426586-382b-4b5e-8f3e-517e370b9521", "content-length": "179", "date": "Wed, 30 Apr 2025 06:44:14 GMT"} })
115982025-04-30T06:44:15.333ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0/component/stage0/caboose?firmware_slot=1
115992025-04-30T06:44:15.340ZDEBGtest_utilization: attempt to fetch latest inventory collection result = Ok("found")
116002025-04-30T06:44:15.345ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "c653d90e-b23a-4628-816f-498c31ffab83", "content-length": "179", "date": "Wed, 30 Apr 2025 06:44:14 GMT"} })
116012025-04-30T06:44:15.345ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0/component/rot/cmpa
116022025-04-30T06:44:15.348ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "b5a5f85e-b6cd-47c8-86e3-81238aede3a9", "content-length": "702", "date": "Wed, 30 Apr 2025 06:44:14 GMT"} })
116032025-04-30T06:44:15.348ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0/component/rot/cfpa
116042025-04-30T06:44:15.349ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "644a0c55-4ca5-4cd7-b7e8-55b13787241d", "content-length": "727", "date": "Wed, 30 Apr 2025 06:44:14 GMT"} })
116052025-04-30T06:44:15.350ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0/component/rot/cfpa
116062025-04-30T06:44:15.351ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "6a91352e-3716-4084-8678-a92e7309aa3f", "content-length": "729", "date": "Wed, 30 Apr 2025 06:44:14 GMT"} })
116072025-04-30T06:44:15.351ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0/component/rot/cfpa
116082025-04-30T06:44:15.353ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "02ec3b40-2b12-4026-9a69-a74ea62acfae", "content-length": "728", "date": "Wed, 30 Apr 2025 06:44:14 GMT"} })
116092025-04-30T06:44:15.353ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1
116102025-04-30T06:44:15.354ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1", status: 200, headers: {"content-type": "application/json", "x-request-id": "69a72b39-d6de-48a1-8cc4-402aaee5c96f", "content-length": "734", "date": "Wed, 30 Apr 2025 06:44:14 GMT"} })
116112025-04-30T06:44:15.355ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1/component/sp/caboose?firmware_slot=0
116122025-04-30T06:44:15.356ZDEBGtest_utilization (results-sink): inserting 1 samples into database collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
116132025-04-30T06:44:15.356ZDEBGtest_utilization (results-sink-cluster): inserting 1 samples into database collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
116142025-04-30T06:44:15.356ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "2b45ffd6-734b-4893-9bde-f2b6f63a6c37", "content-length": "111", "date": "Wed, 30 Apr 2025 06:44:14 GMT"} })
116152025-04-30T06:44:15.356ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1/component/sp/caboose?firmware_slot=1
116162025-04-30T06:44:15.357ZDEBGtest_utilization (clickhouse-client): unrolling 1 total samples collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 4d1485f0-110e-4161-9a33-c4610d68be5d
116172025-04-30T06:44:15.358ZTRACtest_utilization (clickhouse-client): inserting data collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 4d1485f0-110e-4161-9a33-c4610d68be5d n_columns = 4 n_rows = 2 sql = INSERT INTO oximeter.fields_string FORMAT Native
116182025-04-30T06:44:15.360ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "e1e6b670-ad15-437b-bf6f-0b15f4e3b4bd", "content-length": "111", "date": "Wed, 30 Apr 2025 06:44:14 GMT"} })
116192025-04-30T06:44:15.360ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1/component/rot/caboose?firmware_slot=0
116202025-04-30T06:44:15.362ZDEBGtest_utilization (clickhouse-client): inserted rows into table collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 4d1485f0-110e-4161-9a33-c4610d68be5d n_rows = 2 table_name = fields_string
116212025-04-30T06:44:15.362ZTRACtest_utilization (clickhouse-client): inserting data collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 4d1485f0-110e-4161-9a33-c4610d68be5d n_columns = 4 n_rows = 1 sql = INSERT INTO oximeter.measurements_i64 FORMAT Native
116222025-04-30T06:44:15.366ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "2edb9a53-837d-4043-ae3a-1b89200e9a15", "content-length": "170", "date": "Wed, 30 Apr 2025 06:44:14 GMT"} })
116232025-04-30T06:44:15.366ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1/component/rot/caboose?firmware_slot=1
116242025-04-30T06:44:15.367ZDEBGtest_utilization (clickhouse-client): inserted rows into table collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 4d1485f0-110e-4161-9a33-c4610d68be5d n_rows = 1 table_name = measurements_i64
116252025-04-30T06:44:15.367ZTRACtest_utilization (results-sink): successfully inserted samples collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
116262025-04-30T06:44:15.369ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "80891976-b906-44de-8c2c-15db3ad219a6", "content-length": "170", "date": "Wed, 30 Apr 2025 06:44:14 GMT"} })
116272025-04-30T06:44:15.370ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1/component/stage0/caboose?firmware_slot=0
116282025-04-30T06:44:15.373ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "2e8a3aab-765b-440b-b2df-f3c34133816c", "content-length": "179", "date": "Wed, 30 Apr 2025 06:44:14 GMT"} })
116292025-04-30T06:44:15.373ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1/component/stage0/caboose?firmware_slot=1
116302025-04-30T06:44:15.375ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "29e355c9-36f6-4e94-becb-9ad7ca5e58cc", "content-length": "179", "date": "Wed, 30 Apr 2025 06:44:14 GMT"} })
116312025-04-30T06:44:15.376ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1/component/rot/cmpa
116322025-04-30T06:44:15.377ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "fb598cca-8733-457e-ad6d-b27cf07a2d18", "content-length": "702", "date": "Wed, 30 Apr 2025 06:44:14 GMT"} })
116332025-04-30T06:44:15.378ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1/component/rot/cfpa
116342025-04-30T06:44:15.379ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "321a89b9-a789-49d3-9bca-5f6cb164435c", "content-length": "727", "date": "Wed, 30 Apr 2025 06:44:14 GMT"} })
116352025-04-30T06:44:15.379ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1/component/rot/cfpa
116362025-04-30T06:44:15.380ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "0b63cb3a-9f96-40a6-8e03-877ddbf2ad7a", "content-length": "729", "date": "Wed, 30 Apr 2025 06:44:14 GMT"} })
116372025-04-30T06:44:15.381ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1/component/rot/cfpa
116382025-04-30T06:44:15.385ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "c84b8769-24e4-4b8c-981e-cd9bbfd70c5b", "content-length": "728", "date": "Wed, 30 Apr 2025 06:44:14 GMT"} })
116392025-04-30T06:44:15.386ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0
116402025-04-30T06:44:15.390ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "99ea6ddb-0980-421b-a552-22abcd2093d9", "content-length": "723", "date": "Wed, 30 Apr 2025 06:44:14 GMT"} })
116412025-04-30T06:44:15.390ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0/component/sp/caboose?firmware_slot=0
116422025-04-30T06:44:15.391ZTRACtest_utilization (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory
116432025-04-30T06:44:15.392ZTRACtest_utilization (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
116442025-04-30T06:44:15.392ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
116452025-04-30T06:44:15.392ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
116462025-04-30T06:44:15.393ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
116472025-04-30T06:44:15.397ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "7feeda21-1bda-4eea-ad36-99695c1ff0dc", "content-length": "109", "date": "Wed, 30 Apr 2025 06:44:14 GMT"} })
116482025-04-30T06:44:15.397ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0/component/sp/caboose?firmware_slot=1
116492025-04-30T06:44:15.398ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
116502025-04-30T06:44:15.408ZDEBGtest_utilization (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory result = Ok(())
116512025-04-30T06:44:15.408ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
116522025-04-30T06:44:15.408ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
116532025-04-30T06:44:15.408ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
116542025-04-30T06:44:15.411ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
116552025-04-30T06:44:15.411ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
116562025-04-30T06:44:15.412ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
116572025-04-30T06:44:15.413ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "df7910dc-6264-4b77-9444-69b4b47ac619", "content-length": "109", "date": "Wed, 30 Apr 2025 06:44:14 GMT"} })
116582025-04-30T06:44:15.413ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0/component/rot/caboose?firmware_slot=0
116592025-04-30T06:44:15.416ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "92f2985a-8783-483f-8fab-4855c5bcd13a", "content-length": "169", "date": "Wed, 30 Apr 2025 06:44:14 GMT"} })
116602025-04-30T06:44:15.417ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0/component/rot/caboose?firmware_slot=1
116612025-04-30T06:44:15.421ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "6bcb1cc3-4123-47bf-a041-7c55c4d42990", "content-length": "169", "date": "Wed, 30 Apr 2025 06:44:14 GMT"} })
116622025-04-30T06:44:15.422ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0/component/stage0/caboose?firmware_slot=0
116632025-04-30T06:44:15.426ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "4ead70f7-a792-46b1-8435-b88a2596431e", "content-length": "178", "date": "Wed, 30 Apr 2025 06:44:14 GMT"} })
116642025-04-30T06:44:15.427ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0/component/stage0/caboose?firmware_slot=1
116652025-04-30T06:44:15.431ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "90ab3ee2-92bf-41f7-bbbd-4ce0c029218f", "content-length": "178", "date": "Wed, 30 Apr 2025 06:44:14 GMT"} })
116662025-04-30T06:44:15.431ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0/component/rot/cmpa
116672025-04-30T06:44:15.431ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "e49df1e4-7f07-4d89-bfbe-66a1b826e383", "content-length": "702", "date": "Wed, 30 Apr 2025 06:44:14 GMT"} })
116682025-04-30T06:44:15.431ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0/component/rot/cfpa
116692025-04-30T06:44:15.438ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "c1a97562-4c68-445c-99ae-cd0eba1469bc", "content-length": "727", "date": "Wed, 30 Apr 2025 06:44:14 GMT"} })
116702025-04-30T06:44:15.438ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0/component/rot/cfpa
116712025-04-30T06:44:15.445ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "a7a046a2-d662-44bd-bbc8-05af67b0c8ad", "content-length": "729", "date": "Wed, 30 Apr 2025 06:44:14 GMT"} })
116722025-04-30T06:44:15.445ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0/component/rot/cfpa
116732025-04-30T06:44:15.446ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "2705b737-d999-4942-8c51-0666cdff281b", "content-length": "728", "date": "Wed, 30 Apr 2025 06:44:14 GMT"} })
116742025-04-30T06:44:15.447ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1
116752025-04-30T06:44:15.450ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1", status: 200, headers: {"content-type": "application/json", "x-request-id": "444536e3-1545-4eae-9ec4-a4068c74677c", "content-length": "723", "date": "Wed, 30 Apr 2025 06:44:14 GMT"} })
116762025-04-30T06:44:15.451ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1/component/sp/caboose?firmware_slot=0
116772025-04-30T06:44:15.460ZDEBGtest_utilization: attempt to fetch latest inventory collection result = Ok("found")
116782025-04-30T06:44:15.461ZWARNtest_utilization (results-sink-cluster): failed to insert some results into metric DB collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 error = Failed to check out connection to database: No backends found for this service file = oximeter/collector/src/results_sink.rs:92
116792025-04-30T06:44:15.462ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "fa5da379-deab-4694-97c4-aca8412a4ae7", "content-length": "109", "date": "Wed, 30 Apr 2025 06:44:14 GMT"} })
116802025-04-30T06:44:15.462ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1/component/sp/caboose?firmware_slot=1
116812025-04-30T06:44:15.465ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "2ce78019-3c79-4052-9542-2e66a40c1f19", "content-length": "109", "date": "Wed, 30 Apr 2025 06:44:14 GMT"} })
116822025-04-30T06:44:15.465ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1/component/rot/caboose?firmware_slot=0
116832025-04-30T06:44:15.467ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "a2da0fb1-dd68-4518-ba1f-13549bcaf8a1", "content-length": "169", "date": "Wed, 30 Apr 2025 06:44:14 GMT"} })
116842025-04-30T06:44:15.468ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1/component/rot/caboose?firmware_slot=1
116852025-04-30T06:44:15.470ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "e2a35c89-c1d5-4d1b-b1ed-570173add556", "content-length": "169", "date": "Wed, 30 Apr 2025 06:44:14 GMT"} })
116862025-04-30T06:44:15.470ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1/component/stage0/caboose?firmware_slot=0
116872025-04-30T06:44:15.472ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "54704e92-8521-4d75-aa3d-d92084e892d0", "content-length": "178", "date": "Wed, 30 Apr 2025 06:44:14 GMT"} })
116882025-04-30T06:44:15.472ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1/component/stage0/caboose?firmware_slot=1
116892025-04-30T06:44:15.475ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "a563409d-f0ef-44ec-b4e8-5be27997e82a", "content-length": "178", "date": "Wed, 30 Apr 2025 06:44:14 GMT"} })
116902025-04-30T06:44:15.475ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1/component/rot/cmpa
116912025-04-30T06:44:15.476ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "a4f35444-9a5b-4c51-8a49-2c33fcb4553e", "content-length": "702", "date": "Wed, 30 Apr 2025 06:44:14 GMT"} })
116922025-04-30T06:44:15.476ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1/component/rot/cfpa
116932025-04-30T06:44:15.477ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "a8a16533-68c3-4ff6-9ee9-235ebad72a3f", "content-length": "727", "date": "Wed, 30 Apr 2025 06:44:14 GMT"} })
116942025-04-30T06:44:15.477ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1/component/rot/cfpa
116952025-04-30T06:44:15.478ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "14b7a5f8-37f0-4e9d-bd5c-aa332bb87641", "content-length": "729", "date": "Wed, 30 Apr 2025 06:44:14 GMT"} })
116962025-04-30T06:44:15.478ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1/component/rot/cfpa
116972025-04-30T06:44:15.480ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "b491cf77-c19b-4fb7-95f2-8af56e7388ab", "content-length": "728", "date": "Wed, 30 Apr 2025 06:44:14 GMT"} })
116982025-04-30T06:44:15.480ZTRACtest_utilization (BackgroundTasks): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet
116992025-04-30T06:44:15.481ZTRACtest_utilization (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
117002025-04-30T06:44:15.481ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
117012025-04-30T06:44:15.481ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
117022025-04-30T06:44:15.482ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
117032025-04-30T06:44:15.485ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
117042025-04-30T06:44:15.490ZDEBGtest_utilization (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet result = Ok(())
117052025-04-30T06:44:15.490ZTRACtest_utilization (BackgroundTasks): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet
117062025-04-30T06:44:15.490ZTRACtest_utilization (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
117072025-04-30T06:44:15.490ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
117082025-04-30T06:44:15.490ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
117092025-04-30T06:44:15.491ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
117102025-04-30T06:44:15.495ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
117112025-04-30T06:44:15.500ZDEBGtest_utilization (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet result = Ok(())
117122025-04-30T06:44:15.500ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
117132025-04-30T06:44:15.500ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
117142025-04-30T06:44:15.500ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
117152025-04-30T06:44:15.594ZDEBGtest_utilization (BackgroundTasks): begin collection from Sled Agent background_task = inventory_collection sled_agent_url = http://[::1]:39365
117162025-04-30T06:44:15.594ZDEBGtest_utilization (BackgroundTasks): client request SledAgent = http://[::1]:39365 background_task = inventory_collection body = None method = GET uri = http://[::1]:39365/inventory
117172025-04-30T06:44:15.596ZTRACtest_utilization (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory
117182025-04-30T06:44:15.597ZTRACtest_utilization (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
117192025-04-30T06:44:15.597ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
117202025-04-30T06:44:15.597ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
117212025-04-30T06:44:15.600ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
117222025-04-30T06:44:15.600ZINFOtest_utilization (dropshot): accepted connection local_addr = [::1]:39365 remote_addr = [::1]:65131 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
117232025-04-30T06:44:15.603ZTRACtest_utilization (dropshot): incoming request local_addr = [::1]:39365 method = GET remote_addr = [::1]:65131 req_id = 99521baf-12db-491a-95a7-f8812454f931 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /inventory
117242025-04-30T06:44:15.604ZINFOtest_utilization (dropshot): request completed latency_us = 464 local_addr = [::1]:39365 method = GET remote_addr = [::1]:65131 req_id = 99521baf-12db-491a-95a7-f8812454f931 response_code = 200 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /inventory
117252025-04-30T06:44:15.605ZDEBGtest_utilization (BackgroundTasks): client response SledAgent = http://[::1]:39365 background_task = inventory_collection result = Ok(Response { url: "http://[::1]:39365/inventory", status: 200, headers: {"content-type": "application/json", "x-request-id": "99521baf-12db-491a-95a7-f8812454f931", "content-length": "3141", "date": "Wed, 30 Apr 2025 06:44:14 GMT"} })
117262025-04-30T06:44:15.607ZDEBGtest_utilization (BackgroundTasks): begin collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
117272025-04-30T06:44:15.607ZDEBGtest_utilization (BackgroundTasks): end collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
117282025-04-30T06:44:15.607ZDEBGtest_utilization (BackgroundTasks): finished collection background_task = inventory_collection
117292025-04-30T06:44:15.614ZTRACtest_utilization (BackgroundTasks): authorize begin action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory
117302025-04-30T06:44:15.614ZTRACtest_utilization (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
117312025-04-30T06:44:15.614ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
117322025-04-30T06:44:15.614ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
117332025-04-30T06:44:15.615ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
117342025-04-30T06:44:15.617ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
117352025-04-30T06:44:15.626ZDEBGtest_utilization (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory result = Ok(())
117362025-04-30T06:44:15.626ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
117372025-04-30T06:44:15.626ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
117382025-04-30T06:44:15.627ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
117392025-04-30T06:44:15.634ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
117402025-04-30T06:44:15.639ZDEBGtest_utilization (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory result = Ok(())
117412025-04-30T06:44:15.639ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
117422025-04-30T06:44:15.639ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
117432025-04-30T06:44:15.640ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
117442025-04-30T06:44:15.643ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
117452025-04-30T06:44:15.643ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
117462025-04-30T06:44:15.643ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
117472025-04-30T06:44:15.697ZWARNtest_utilization (datastore): Non-retryable transaction failure err = DatabaseError(UniqueViolation, "duplicate key value violates unique constraint \\"inv_caboose_pkey\\"") transaction = inventory_insert_collection
117482025-04-30T06:44:15.697ZWARNtest_utilization (BackgroundTasks): inventory collection failed background_task = inventory_collection error = failed to collect inventory: saving inventory to database: Internal Error: unexpected database error: duplicate key value violates unique constraint "inv_caboose_pkey"
117492025-04-30T06:44:15.697ZDEBGtest_utilization (BackgroundTasks): activation complete background_task = inventory_collection elapsed = 490.741751ms iteration = 28 status = {"error":"failed to collect inventory: saving inventory to database: Internal Error: unexpected database error: duplicate key value violates unique constraint \\"inv_caboose_pkey\\""}
117502025-04-30T06:44:15.697ZDEBGtest_utilization (BackgroundTasks): activating background_task = inventory_collection iteration = 29 reason = Signaled
117512025-04-30T06:44:15.697ZTRACtest_utilization (BackgroundTasks): authorize begin action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory
117522025-04-30T06:44:15.697ZTRACtest_utilization (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
117532025-04-30T06:44:15.697ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
117542025-04-30T06:44:15.697ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
117552025-04-30T06:44:15.700ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
117562025-04-30T06:44:15.709ZDEBGtest_utilization: attempt to fetch latest inventory collection result = Ok("found")
117572025-04-30T06:44:15.709ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
117582025-04-30T06:44:15.717ZDEBGtest_utilization (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory result = Ok(())
117592025-04-30T06:44:15.718ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
117602025-04-30T06:44:15.718ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
117612025-04-30T06:44:15.720ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
117622025-04-30T06:44:15.724ZDEBGtest_utilization (BackgroundTasks): inventory_prune_one: nothing eligible for removal (too few) background_task = inventory_collection candidates = [(13d8b8d9-6c6b-47a6-9190-952544edb75b, 0)]
117632025-04-30T06:44:15.724ZTRACtest_utilization (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal
117642025-04-30T06:44:15.726ZTRACtest_utilization (dns): buffer buffer = Length: 67 (0x43) bytes\n0000: 70 26 01 00 00 01 00 00 00 00 00 01 04 5f 6d 67 p&..........._mg\n0010: 73 04 5f 74 63 70 0d 63 6f 6e 74 72 6f 6c 2d 70 s._tcp.control-p\n0020: 6c 61 6e 65 05 6f 78 69 64 65 08 69 6e 74 65 72 lane.oxide.inter\n0030: 6e 61 6c 00 00 21 00 01 00 00 29 04 d0 00 00 00 nal..!....).....\n0040: 00 00 00 ... kind = dns peer_addr = [::1]:60314 req_id = e83d06cb-7b65-49f4-bbbf-b1dd83edddc1
117652025-04-30T06:44:15.727ZDEBGtest_utilization (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 28710,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_mgs._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_mgs._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 4,\n 95,\n 109,\n 103,\n 115,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:60314 req_id = e83d06cb-7b65-49f4-bbbf-b1dd83edddc1
117662025-04-30T06:44:15.727ZDEBGtest_utilization (store): query key key = _mgs._tcp
117672025-04-30T06:44:15.729ZDEBGtest_utilization (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
117682025-04-30T06:44:15.729ZDEBGtest_utilization (dns): dns response additional_records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal"), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] kind = dns peer_addr = [::1]:60314 query = LowerQuery { name: LowerName(Name("_mgs._tcp.control-plane.oxide.internal.")), original: Query { name: Name("_mgs._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN } } records = [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 48759, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal") })) }] req_id = e83d06cb-7b65-49f4-bbbf-b1dd83edddc1
117692025-04-30T06:44:15.731ZDEBGtest_utilization (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgs._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 48759, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }], valid_until: Instant { tv_sec: 4320, tv_nsec: 529888987 } })
117702025-04-30T06:44:15.731ZTRACtest_utilization (DnsResolver): lookup_service_targets: looking up SRV target
117712025-04-30T06:44:15.731ZTRACtest_utilization (dns): buffer buffer = Length: 108 (0x6c) bytes\n0000: 05 12 01 00 00 01 00 00 00 00 00 01 2d 64 65 6e ............-den\n0010: 64 72 69 74 65 2d 62 36 64 36 35 33 34 31 2d 31 drite-b6d65341-1\n0020: 36 37 63 2d 34 31 64 66 2d 39 62 35 63 2d 34 31 67c-41df-9b5c-41\n0030: 63 64 65 64 39 39 63 32 32 39 04 68 6f 73 74 0d cded99c229.host.\n0040: 63 6f 6e 74 72 6f 6c 2d 70 6c 61 6e 65 05 6f 78 control-plane.ox\n0050: 69 64 65 08 69 6e 74 65 72 6e 61 6c 00 00 1c 00 ide.internal....\n0060: 01 00 00 29 04 d0 00 00 00 00 00 00 ...)........ kind = dns peer_addr = [::1]:55834 req_id = 75f0df80-1d4d-488b-ad6c-2deca66c4eeb
117722025-04-30T06:44:15.731ZDEBGtest_utilization (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 1298,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n query_type: AAAA,\n query_class: IN,\n },\n },\n original: [\n 45,\n 100,\n 101,\n 110,\n 100,\n 114,\n 105,\n 116,\n 101,\n 45,\n 98,\n 54,\n 100,\n 54,\n 53,\n 51,\n 52,\n 49,\n 45,\n 49,\n 54,\n 55,\n 99,\n 45,\n 52,\n 49,\n 100,\n 102,\n 45,\n 57,\n 98,\n 53,\n 99,\n 45,\n 52,\n 49,\n 99,\n 100,\n 101,\n 100,\n 57,\n 57,\n 99,\n 50,\n 50,\n 57,\n 4,\n 104,\n 111,\n 115,\n 116,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 28,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:55834 req_id = 75f0df80-1d4d-488b-ad6c-2deca66c4eeb
117732025-04-30T06:44:15.731ZDEBGtest_utilization (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
117742025-04-30T06:44:15.731ZDEBGtest_utilization (dns): dns response additional_records = [] kind = dns peer_addr = [::1]:55834 query = LowerQuery { name: LowerName(Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.")), original: Query { name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), query_type: AAAA, query_class: IN } } records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] req_id = 75f0df80-1d4d-488b-ad6c-2deca66c4eeb
117752025-04-30T06:44:15.831ZTRACtest_utilization (DnsResolver): lookup_all_socket_v6 srv dns_name = _clickhouse-admin-keeper._tcp.control-plane.oxide.internal
117762025-04-30T06:44:15.831ZTRACtest_utilization (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory
117772025-04-30T06:44:15.831ZTRACtest_utilization (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
117782025-04-30T06:44:15.831ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
117792025-04-30T06:44:15.831ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
117802025-04-30T06:44:15.832ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
117812025-04-30T06:44:15.833ZTRACtest_utilization (dns): buffer buffer = Length: 87 (0x57) bytes\n0000: 52 54 01 00 00 01 00 00 00 00 00 01 18 5f 63 6c RT..........._cl\n0010: 69 63 6b 68 6f 75 73 65 2d 61 64 6d 69 6e 2d 6b ickhouse-admin-k\n0020: 65 65 70 65 72 04 5f 74 63 70 0d 63 6f 6e 74 72 eeper._tcp.contr\n0030: 6f 6c 2d 70 6c 61 6e 65 05 6f 78 69 64 65 08 69 ol-plane.oxide.i\n0040: 6e 74 65 72 6e 61 6c 00 00 21 00 01 00 00 29 04 nternal..!....).\n0050: d0 00 00 00 00 00 00 ....... kind = dns peer_addr = [::1]:50702 req_id = baaa9e4b-bdcf-4249-aeb4-dc9b421d95f3
117822025-04-30T06:44:15.834ZDEBGtest_utilization (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 21076,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 24,\n 95,\n 99,\n 108,\n 105,\n 99,\n 107,\n 104,\n 111,\n 117,\n 115,\n 101,\n 45,\n 97,\n 100,\n 109,\n 105,\n 110,\n 45,\n 107,\n 101,\n 101,\n 112,\n 101,\n 114,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:50702 req_id = baaa9e4b-bdcf-4249-aeb4-dc9b421d95f3
117832025-04-30T06:44:15.834ZDEBGtest_utilization (store): query key key = _clickhouse-admin-keeper._tcp
117842025-04-30T06:44:15.834ZERROtest_utilization (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 21076, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 1, }, query: WireQuery { query: LowerQuery { name: LowerName( Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), ), original: Query { name: Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN, }, }, original: [ 24, 95, 99, 108, 105, 99, 107, 104, 111, 117, 115, 101, 45, 97, 100, 109, 105, 110, 45, 107, 101, 101, 112, 101, 114, 4, 95, 116, 99, 112, 13, 99, 111, 110, 116, 114, 111, 108, 45, 112, 108, 97, 110, 101, 5, 111, 120, 105, 100, 101, 8, 105, 110, 116, 101, 114, 110, 97, 108, 0, 0, 33, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: Some( Edns { rcode_high: 0, version: 0, dnssec_ok: false, max_payload: 1232, options: OPT { options: {}, }, }, ), } NXDOMAIN: no records found for name: "_clickhouse-admin-keeper._tcp.control-plane.oxide.internal." kind = dns peer_addr = [::1]:50702 req_id = baaa9e4b-bdcf-4249-aeb4-dc9b421d95f3
117852025-04-30T06:44:15.834ZDEBGtest_utilization (BackgroundTasks): begin collection background_task = inventory_collection
117862025-04-30T06:44:15.835ZDEBGtest_utilization (BackgroundTasks): begin collection from MGS background_task = inventory_collection mgs_url = http://[::1]:48759
117872025-04-30T06:44:15.835ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/ignition
117882025-04-30T06:44:15.836ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
117892025-04-30T06:44:15.842ZDEBGtest_utilization (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory result = Ok(())
117902025-04-30T06:44:15.842ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
117912025-04-30T06:44:15.842ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
117922025-04-30T06:44:15.843ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
117932025-04-30T06:44:15.846ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/ignition", status: 200, headers: {"content-type": "application/json", "x-request-id": "c5018878-0dc0-42eb-abd1-f1eab8f58731", "content-length": "835", "date": "Wed, 30 Apr 2025 06:44:14 GMT"} })
117942025-04-30T06:44:15.846ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0
117952025-04-30T06:44:15.849ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "45242c85-3398-41e7-8175-5e39cd9b4d3f", "content-length": "734", "date": "Wed, 30 Apr 2025 06:44:14 GMT"} })
117962025-04-30T06:44:15.849ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0/component/sp/caboose?firmware_slot=0
117972025-04-30T06:44:15.851ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
117982025-04-30T06:44:15.851ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
117992025-04-30T06:44:15.851ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
118002025-04-30T06:44:15.854ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "4fb3208a-6d5f-4719-b32b-0ed3f740820b", "content-length": "111", "date": "Wed, 30 Apr 2025 06:44:14 GMT"} })
118012025-04-30T06:44:15.854ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0/component/sp/caboose?firmware_slot=1
118022025-04-30T06:44:15.858ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "e132f560-f84c-4dea-87f8-dd692d920332", "content-length": "111", "date": "Wed, 30 Apr 2025 06:44:14 GMT"} })
118032025-04-30T06:44:15.858ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0/component/rot/caboose?firmware_slot=0
118042025-04-30T06:44:15.863ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "b5e9242a-5f0c-4a0c-8527-4a3edced2315", "content-length": "170", "date": "Wed, 30 Apr 2025 06:44:14 GMT"} })
118052025-04-30T06:44:15.863ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0/component/rot/caboose?firmware_slot=1
118062025-04-30T06:44:15.867ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "24fc9d36-bb59-42cb-a1c5-cc847d4a935c", "content-length": "170", "date": "Wed, 30 Apr 2025 06:44:14 GMT"} })
118072025-04-30T06:44:15.867ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0/component/stage0/caboose?firmware_slot=0
118082025-04-30T06:44:15.870ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "17708fdc-4cbc-4545-94cc-f02a9c65b34b", "content-length": "179", "date": "Wed, 30 Apr 2025 06:44:14 GMT"} })
118092025-04-30T06:44:15.871ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0/component/stage0/caboose?firmware_slot=1
118102025-04-30T06:44:15.874ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "f64050aa-42db-46c6-9f11-b342b913b2da", "content-length": "179", "date": "Wed, 30 Apr 2025 06:44:14 GMT"} })
118112025-04-30T06:44:15.875ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0/component/rot/cmpa
118122025-04-30T06:44:15.876ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "6d314a21-f73d-4752-866b-a5426359816b", "content-length": "702", "date": "Wed, 30 Apr 2025 06:44:14 GMT"} })
118132025-04-30T06:44:15.876ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0/component/rot/cfpa
118142025-04-30T06:44:15.878ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "3bc5267a-3abd-4354-a9e0-2791e1abae3a", "content-length": "727", "date": "Wed, 30 Apr 2025 06:44:14 GMT"} })
118152025-04-30T06:44:15.878ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0/component/rot/cfpa
118162025-04-30T06:44:15.880ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "407635f4-f000-4886-827e-f109d0f27be1", "content-length": "729", "date": "Wed, 30 Apr 2025 06:44:14 GMT"} })
118172025-04-30T06:44:15.881ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0/component/rot/cfpa
118182025-04-30T06:44:15.882ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "c175f07c-94d2-42d2-acc0-10d5b75e46c6", "content-length": "728", "date": "Wed, 30 Apr 2025 06:44:15 GMT"} })
118192025-04-30T06:44:15.883ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1
118202025-04-30T06:44:15.884ZDEBGtest_utilization: attempt to fetch latest inventory collection result = Ok("found")
118212025-04-30T06:44:15.887ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1", status: 200, headers: {"content-type": "application/json", "x-request-id": "ed472eb5-0867-4bd2-b56f-6d61b88c77a3", "content-length": "734", "date": "Wed, 30 Apr 2025 06:44:15 GMT"} })
118222025-04-30T06:44:15.887ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1/component/sp/caboose?firmware_slot=0
118232025-04-30T06:44:15.892ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "05aa2ac7-b387-4652-818e-8453e2c8e130", "content-length": "111", "date": "Wed, 30 Apr 2025 06:44:15 GMT"} })
118242025-04-30T06:44:15.892ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1/component/sp/caboose?firmware_slot=1
118252025-04-30T06:44:15.896ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "70ddf1e1-c171-4d12-aebc-ccb06f6132f6", "content-length": "111", "date": "Wed, 30 Apr 2025 06:44:15 GMT"} })
118262025-04-30T06:44:15.896ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1/component/rot/caboose?firmware_slot=0
118272025-04-30T06:44:15.901ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "7e4c72bf-9d62-4608-9f94-7a87c8df5fe2", "content-length": "170", "date": "Wed, 30 Apr 2025 06:44:15 GMT"} })
118282025-04-30T06:44:15.901ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1/component/rot/caboose?firmware_slot=1
118292025-04-30T06:44:15.906ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "c2ef9a18-954e-432e-b602-b2e3e2e84891", "content-length": "170", "date": "Wed, 30 Apr 2025 06:44:15 GMT"} })
118302025-04-30T06:44:15.907ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1/component/stage0/caboose?firmware_slot=0
118312025-04-30T06:44:15.913ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "e43eb1bf-bebf-4c20-99d0-dedc204e62b6", "content-length": "179", "date": "Wed, 30 Apr 2025 06:44:15 GMT"} })
118322025-04-30T06:44:15.913ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1/component/stage0/caboose?firmware_slot=1
118332025-04-30T06:44:15.918ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "2bbe6795-6965-4055-a0b1-2e8feaa3adec", "content-length": "179", "date": "Wed, 30 Apr 2025 06:44:15 GMT"} })
118342025-04-30T06:44:15.918ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1/component/rot/cmpa
118352025-04-30T06:44:15.921ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "f64ab3dc-d00d-427c-8046-365484d982f4", "content-length": "702", "date": "Wed, 30 Apr 2025 06:44:15 GMT"} })
118362025-04-30T06:44:15.921ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1/component/rot/cfpa
118372025-04-30T06:44:15.923ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "a1d6d8f0-d099-470b-8a6d-8cdbe534cec3", "content-length": "727", "date": "Wed, 30 Apr 2025 06:44:15 GMT"} })
118382025-04-30T06:44:15.923ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1/component/rot/cfpa
118392025-04-30T06:44:15.927ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "1a5c47b9-813d-4c18-99ac-a843d5f1e2ef", "content-length": "729", "date": "Wed, 30 Apr 2025 06:44:15 GMT"} })
118402025-04-30T06:44:15.927ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1/component/rot/cfpa
118412025-04-30T06:44:15.930ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "1df6a332-3590-48d0-84a2-b1bfe6580be3", "content-length": "728", "date": "Wed, 30 Apr 2025 06:44:15 GMT"} })
118422025-04-30T06:44:15.931ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0
118432025-04-30T06:44:15.934ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "d91bed8b-ed63-4cc9-94c9-8ecf2e1031a9", "content-length": "723", "date": "Wed, 30 Apr 2025 06:44:15 GMT"} })
118442025-04-30T06:44:15.934ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0/component/sp/caboose?firmware_slot=0
118452025-04-30T06:44:15.935ZTRACtest_utilization (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory
118462025-04-30T06:44:15.935ZTRACtest_utilization (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
118472025-04-30T06:44:15.936ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
118482025-04-30T06:44:15.936ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
118492025-04-30T06:44:15.936ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
118502025-04-30T06:44:15.942ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
118512025-04-30T06:44:15.955ZDEBGtest_utilization (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory result = Ok(())
118522025-04-30T06:44:15.955ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
118532025-04-30T06:44:15.955ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
118542025-04-30T06:44:15.956ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
118552025-04-30T06:44:15.957ZINFOtest_utilization (oximeter-agent): refreshing list of producers from Nexus collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 file = oximeter/collector/src/agent.rs:562
118562025-04-30T06:44:15.957ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "1e1baac3-928e-47c3-b5e8-a9e8205312f9", "content-length": "109", "date": "Wed, 30 Apr 2025 06:44:15 GMT"} })
118572025-04-30T06:44:15.957ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0/component/sp/caboose?firmware_slot=1
118582025-04-30T06:44:15.958ZDEBGtest_utilization (oximeter): client request body = None method = GET uri = http://127.0.0.1:32956/metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?sort_by=id_ascending
118592025-04-30T06:44:15.959ZTRACtest_utilization (dropshot_internal): incoming request local_addr = 127.0.0.1:32956 method = GET remote_addr = 127.0.0.1:55797 req_id = 7fd47fb0-9b26-42a5-984b-671602c00435 uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?sort_by=id_ascending
118602025-04-30T06:44:15.959ZTRACtest_utilization (dropshot_internal): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = 127.0.0.1:32956 method = GET remote_addr = 127.0.0.1:55797 req_id = 7fd47fb0-9b26-42a5-984b-671602c00435 resource = Database uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?sort_by=id_ascending
118612025-04-30T06:44:15.959ZDEBGtest_utilization (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = 127.0.0.1:32956 method = GET remote_addr = 127.0.0.1:55797 req_id = 7fd47fb0-9b26-42a5-984b-671602c00435 roles = RoleSet { roles: {} } uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?sort_by=id_ascending
118622025-04-30T06:44:15.962ZDEBGtest_utilization (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = 127.0.0.1:32956 method = GET remote_addr = 127.0.0.1:55797 req_id = 7fd47fb0-9b26-42a5-984b-671602c00435 resource = Database result = Ok(()) uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?sort_by=id_ascending
118632025-04-30T06:44:15.962ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
118642025-04-30T06:44:15.962ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
118652025-04-30T06:44:15.963ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
118662025-04-30T06:44:15.966ZINFOtest_utilization (dropshot_internal): request completed latency_us = 7499 local_addr = 127.0.0.1:32956 method = GET remote_addr = 127.0.0.1:55797 req_id = 7fd47fb0-9b26-42a5-984b-671602c00435 response_code = 200 uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?sort_by=id_ascending
118672025-04-30T06:44:15.967ZDEBGtest_utilization (oximeter): client response result = Ok(Response { url: "http://127.0.0.1:32956/metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?sort_by=id_ascending", status: 200, headers: {"content-type": "application/json", "x-request-id": "7fd47fb0-9b26-42a5-984b-671602c00435", "content-length": "405", "date": "Wed, 30 Apr 2025 06:44:15 GMT"} })
118682025-04-30T06:44:15.968ZDEBGtest_utilization (oximeter): client request body = None method = GET uri = http://127.0.0.1:32956/metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJhYzU1ZDhkOS1hMjk4LTQ1MzQtODk4Yi1iZWVlMWM5ZTRlNTIifX0%3D
118692025-04-30T06:44:15.970ZTRACtest_utilization (dropshot_internal): incoming request local_addr = 127.0.0.1:32956 method = GET remote_addr = 127.0.0.1:55797 req_id = cf47fb43-068b-4689-819b-dd757333ef32 uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJhYzU1ZDhkOS1hMjk4LTQ1MzQtODk4Yi1iZWVlMWM5ZTRlNTIifX0%3D
118702025-04-30T06:44:15.971ZTRACtest_utilization (dropshot_internal): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = 127.0.0.1:32956 method = GET remote_addr = 127.0.0.1:55797 req_id = cf47fb43-068b-4689-819b-dd757333ef32 resource = Database uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJhYzU1ZDhkOS1hMjk4LTQ1MzQtODk4Yi1iZWVlMWM5ZTRlNTIifX0%3D
118712025-04-30T06:44:15.971ZDEBGtest_utilization (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = 127.0.0.1:32956 method = GET remote_addr = 127.0.0.1:55797 req_id = cf47fb43-068b-4689-819b-dd757333ef32 roles = RoleSet { roles: {} } uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJhYzU1ZDhkOS1hMjk4LTQ1MzQtODk4Yi1iZWVlMWM5ZTRlNTIifX0%3D
118722025-04-30T06:44:15.972ZDEBGtest_utilization (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = 127.0.0.1:32956 method = GET remote_addr = 127.0.0.1:55797 req_id = cf47fb43-068b-4689-819b-dd757333ef32 resource = Database result = Ok(()) uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJhYzU1ZDhkOS1hMjk4LTQ1MzQtODk4Yi1iZWVlMWM5ZTRlNTIifX0%3D
118732025-04-30T06:44:15.980ZINFOtest_utilization (dropshot_internal): request completed latency_us = 10226 local_addr = 127.0.0.1:32956 method = GET remote_addr = 127.0.0.1:55797 req_id = cf47fb43-068b-4689-819b-dd757333ef32 response_code = 200 uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJhYzU1ZDhkOS1hMjk4LTQ1MzQtODk4Yi1iZWVlMWM5ZTRlNTIifX0%3D
118742025-04-30T06:44:15.981ZDEBGtest_utilization (oximeter): client response result = Ok(Response { url: "http://127.0.0.1:32956/metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJhYzU1ZDhkOS1hMjk4LTQ1MzQtODk4Yi1iZWVlMWM5ZTRlNTIifX0%3D", status: 200, headers: {"content-type": "application/json", "x-request-id": "cf47fb43-068b-4689-819b-dd757333ef32", "content-length": "29", "date": "Wed, 30 Apr 2025 06:44:15 GMT"} })
118752025-04-30T06:44:15.981ZTRACtest_utilization (oximeter-agent): ignoring request to update existing metric producer, since the endpoint information is the same as the existing collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
118762025-04-30T06:44:15.981ZTRACtest_utilization (oximeter-agent): ignoring request to update existing metric producer, since the endpoint information is the same as the existing collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = ac55d8d9-a298-4534-898b-beee1c9e4e52
118772025-04-30T06:44:15.981ZINFOtest_utilization (oximeter-agent): refreshed list of producers from Nexus collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 file = oximeter/collector/src/agent.rs:637 n_current_tasks = 2 n_pruned_tasks = 0
118782025-04-30T06:44:15.981ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "1bdeb52e-8069-455e-b79e-3718b5ba7e26", "content-length": "109", "date": "Wed, 30 Apr 2025 06:44:15 GMT"} })
118792025-04-30T06:44:15.982ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0/component/rot/caboose?firmware_slot=0
118802025-04-30T06:44:15.988ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "15c3364a-2e7c-4e4a-b93d-76394f6a7120", "content-length": "169", "date": "Wed, 30 Apr 2025 06:44:15 GMT"} })
118812025-04-30T06:44:15.990ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0/component/rot/caboose?firmware_slot=1
118822025-04-30T06:44:15.996ZDEBGtest_utilization (collection-task): sent timer-based collection request to the collection task collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
118832025-04-30T06:44:15.997ZDEBGtest_utilization (collection-task): collecting from producer collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
118842025-04-30T06:44:16.002ZTRACtest_utilization (collection-task): sent collection request to producer collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
118852025-04-30T06:44:16.003ZDEBGtest_utilization (collection-task): collected results from producer collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 n_results = 1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
118862025-04-30T06:44:16.003ZTRACtest_utilization (collection-task): forwarded results to main collection loop collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
118872025-04-30T06:44:16.003ZTRACtest_utilization (collection-task): top of inner collection loop, waiting for next request collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
118882025-04-30T06:44:16.005ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "e5c15bdb-2571-48d7-bb17-a6a674aa3599", "content-length": "169", "date": "Wed, 30 Apr 2025 06:44:15 GMT"} })
118892025-04-30T06:44:16.005ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0/component/stage0/caboose?firmware_slot=0
118902025-04-30T06:44:16.009ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "4caad686-1a48-4c91-8384-236e3e01eb9e", "content-length": "178", "date": "Wed, 30 Apr 2025 06:44:15 GMT"} })
118912025-04-30T06:44:16.010ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0/component/stage0/caboose?firmware_slot=1
118922025-04-30T06:44:16.014ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "28094c7c-d53b-491b-823b-d49c8c06e4b6", "content-length": "178", "date": "Wed, 30 Apr 2025 06:44:15 GMT"} })
118932025-04-30T06:44:16.014ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0/component/rot/cmpa
118942025-04-30T06:44:16.015ZDEBGtest_utilization: attempt to fetch latest inventory collection result = Ok("found")
118952025-04-30T06:44:16.016ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "1f623f3c-e0ea-4f25-b531-682d36c72ea1", "content-length": "702", "date": "Wed, 30 Apr 2025 06:44:15 GMT"} })
118962025-04-30T06:44:16.016ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0/component/rot/cfpa
118972025-04-30T06:44:16.018ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "edd88689-a6ba-4e6a-b0d4-607f2fc996d0", "content-length": "727", "date": "Wed, 30 Apr 2025 06:44:15 GMT"} })
118982025-04-30T06:44:16.018ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0/component/rot/cfpa
118992025-04-30T06:44:16.019ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "c196ce70-987b-4550-b9b7-a52bbbdb3cf9", "content-length": "729", "date": "Wed, 30 Apr 2025 06:44:15 GMT"} })
119002025-04-30T06:44:16.020ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0/component/rot/cfpa
119012025-04-30T06:44:16.021ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "193a866c-7b4e-4f17-ae64-bbd426099359", "content-length": "728", "date": "Wed, 30 Apr 2025 06:44:15 GMT"} })
119022025-04-30T06:44:16.021ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1
119032025-04-30T06:44:16.023ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1", status: 200, headers: {"content-type": "application/json", "x-request-id": "e6f44bd7-22e5-4d64-99da-5d6ddf26b50b", "content-length": "723", "date": "Wed, 30 Apr 2025 06:44:15 GMT"} })
119042025-04-30T06:44:16.023ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1/component/sp/caboose?firmware_slot=0
119052025-04-30T06:44:16.025ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "e9b518cc-b657-4d4d-9c78-36a3030dd05d", "content-length": "109", "date": "Wed, 30 Apr 2025 06:44:15 GMT"} })
119062025-04-30T06:44:16.026ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1/component/sp/caboose?firmware_slot=1
119072025-04-30T06:44:16.028ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "697e2793-68a0-4ff7-b2f3-522106d268a6", "content-length": "109", "date": "Wed, 30 Apr 2025 06:44:15 GMT"} })
119082025-04-30T06:44:16.028ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1/component/rot/caboose?firmware_slot=0
119092025-04-30T06:44:16.031ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "87b77492-edd1-4fbd-94f3-cb91cf7a77ca", "content-length": "169", "date": "Wed, 30 Apr 2025 06:44:15 GMT"} })
119102025-04-30T06:44:16.032ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1/component/rot/caboose?firmware_slot=1
119112025-04-30T06:44:16.034ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "503c1175-79a9-4eb4-a627-fa39c5088314", "content-length": "169", "date": "Wed, 30 Apr 2025 06:44:15 GMT"} })
119122025-04-30T06:44:16.034ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1/component/stage0/caboose?firmware_slot=0
119132025-04-30T06:44:16.037ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "b54ef347-a998-4eb9-8222-732e3870a5fc", "content-length": "178", "date": "Wed, 30 Apr 2025 06:44:15 GMT"} })
119142025-04-30T06:44:16.037ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1/component/stage0/caboose?firmware_slot=1
119152025-04-30T06:44:16.040ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "22e80bda-0efb-4b8e-bdb7-ad7741f368ed", "content-length": "178", "date": "Wed, 30 Apr 2025 06:44:15 GMT"} })
119162025-04-30T06:44:16.040ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1/component/rot/cmpa
119172025-04-30T06:44:16.041ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "5788e418-e98a-462c-830d-d6bec8c31de0", "content-length": "702", "date": "Wed, 30 Apr 2025 06:44:15 GMT"} })
119182025-04-30T06:44:16.042ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1/component/rot/cfpa
119192025-04-30T06:44:16.043ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "d8169a73-8e65-435f-9480-752d58441cda", "content-length": "727", "date": "Wed, 30 Apr 2025 06:44:15 GMT"} })
119202025-04-30T06:44:16.043ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1/component/rot/cfpa
119212025-04-30T06:44:16.044ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "df185c57-54a6-4686-94b8-cecc8da2fc34", "content-length": "729", "date": "Wed, 30 Apr 2025 06:44:15 GMT"} })
119222025-04-30T06:44:16.044ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1/component/rot/cfpa
119232025-04-30T06:44:16.046ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "c21ec41f-d035-4f6a-a646-8fe367d9373a", "content-length": "728", "date": "Wed, 30 Apr 2025 06:44:15 GMT"} })
119242025-04-30T06:44:16.046ZTRACtest_utilization (BackgroundTasks): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet
119252025-04-30T06:44:16.046ZTRACtest_utilization (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
119262025-04-30T06:44:16.046ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
119272025-04-30T06:44:16.046ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
119282025-04-30T06:44:16.047ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
119292025-04-30T06:44:16.054ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
119302025-04-30T06:44:16.059ZDEBGtest_utilization (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet result = Ok(())
119312025-04-30T06:44:16.059ZTRACtest_utilization (BackgroundTasks): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet
119322025-04-30T06:44:16.059ZTRACtest_utilization (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
119332025-04-30T06:44:16.059ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
119342025-04-30T06:44:16.059ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
119352025-04-30T06:44:16.060ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
119362025-04-30T06:44:16.063ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
119372025-04-30T06:44:16.069ZDEBGtest_utilization (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet result = Ok(())
119382025-04-30T06:44:16.069ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
119392025-04-30T06:44:16.069ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
119402025-04-30T06:44:16.070ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
119412025-04-30T06:44:16.070ZTRACtest_utilization (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory
119422025-04-30T06:44:16.070ZTRACtest_utilization (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
119432025-04-30T06:44:16.070ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
119442025-04-30T06:44:16.070ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
119452025-04-30T06:44:16.071ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
119462025-04-30T06:44:16.074ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
119472025-04-30T06:44:16.082ZDEBGtest_utilization (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory result = Ok(())
119482025-04-30T06:44:16.082ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
119492025-04-30T06:44:16.082ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
119502025-04-30T06:44:16.083ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
119512025-04-30T06:44:16.171ZDEBGtest_utilization (BackgroundTasks): begin collection from Sled Agent background_task = inventory_collection sled_agent_url = http://[::1]:39365
119522025-04-30T06:44:16.171ZDEBGtest_utilization (BackgroundTasks): client request SledAgent = http://[::1]:39365 background_task = inventory_collection body = None method = GET uri = http://[::1]:39365/inventory
119532025-04-30T06:44:16.171ZINFOtest_utilization (dropshot): accepted connection local_addr = [::1]:39365 remote_addr = [::1]:55579 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
119542025-04-30T06:44:16.172ZTRACtest_utilization (dropshot): incoming request local_addr = [::1]:39365 method = GET remote_addr = [::1]:55579 req_id = d2c7d8c3-953f-4bb5-9ec6-0d69d08f644d sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /inventory
119552025-04-30T06:44:16.172ZINFOtest_utilization (dropshot): request completed latency_us = 470 local_addr = [::1]:39365 method = GET remote_addr = [::1]:55579 req_id = d2c7d8c3-953f-4bb5-9ec6-0d69d08f644d response_code = 200 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /inventory
119562025-04-30T06:44:16.172ZDEBGtest_utilization (BackgroundTasks): client response SledAgent = http://[::1]:39365 background_task = inventory_collection result = Ok(Response { url: "http://[::1]:39365/inventory", status: 200, headers: {"content-type": "application/json", "x-request-id": "d2c7d8c3-953f-4bb5-9ec6-0d69d08f644d", "content-length": "3141", "date": "Wed, 30 Apr 2025 06:44:15 GMT"} })
119572025-04-30T06:44:16.174ZDEBGtest_utilization (BackgroundTasks): begin collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
119582025-04-30T06:44:16.174ZDEBGtest_utilization (BackgroundTasks): end collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
119592025-04-30T06:44:16.174ZDEBGtest_utilization (BackgroundTasks): finished collection background_task = inventory_collection
119602025-04-30T06:44:16.177ZTRACtest_utilization (BackgroundTasks): authorize begin action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory
119612025-04-30T06:44:16.177ZTRACtest_utilization (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
119622025-04-30T06:44:16.177ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
119632025-04-30T06:44:16.177ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
119642025-04-30T06:44:16.178ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
119652025-04-30T06:44:16.179ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
119662025-04-30T06:44:16.179ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
119672025-04-30T06:44:16.180ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
119682025-04-30T06:44:16.182ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
119692025-04-30T06:44:16.187ZDEBGtest_utilization (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory result = Ok(())
119702025-04-30T06:44:16.187ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
119712025-04-30T06:44:16.187ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
119722025-04-30T06:44:16.188ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
119732025-04-30T06:44:16.252ZDEBGtest_utilization: attempt to fetch latest inventory collection result = Ok("found")
119742025-04-30T06:44:16.261ZWARNtest_utilization (datastore): Non-retryable transaction failure err = DatabaseError(UniqueViolation, "duplicate key value violates unique constraint \\"inv_caboose_pkey\\"") transaction = inventory_insert_collection
119752025-04-30T06:44:16.262ZWARNtest_utilization (BackgroundTasks): inventory collection failed background_task = inventory_collection error = failed to collect inventory: saving inventory to database: Internal Error: unexpected database error: duplicate key value violates unique constraint "inv_caboose_pkey"
119762025-04-30T06:44:16.264ZDEBGtest_utilization (BackgroundTasks): activation complete background_task = inventory_collection elapsed = 564.991509ms iteration = 29 status = {"error":"failed to collect inventory: saving inventory to database: Internal Error: unexpected database error: duplicate key value violates unique constraint \\"inv_caboose_pkey\\""}
119772025-04-30T06:44:16.264ZDEBGtest_utilization (BackgroundTasks): activating background_task = inventory_collection iteration = 30 reason = Signaled
119782025-04-30T06:44:16.264ZTRACtest_utilization (BackgroundTasks): authorize begin action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory
119792025-04-30T06:44:16.264ZTRACtest_utilization (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
119802025-04-30T06:44:16.264ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
119812025-04-30T06:44:16.264ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
119822025-04-30T06:44:16.265ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
119832025-04-30T06:44:16.270ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
119842025-04-30T06:44:16.288ZDEBGtest_utilization (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory result = Ok(())
119852025-04-30T06:44:16.288ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
119862025-04-30T06:44:16.288ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
119872025-04-30T06:44:16.290ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
119882025-04-30T06:44:16.296ZDEBGtest_utilization (BackgroundTasks): inventory_prune_one: nothing eligible for removal (too few) background_task = inventory_collection candidates = [(13d8b8d9-6c6b-47a6-9190-952544edb75b, 0)]
119892025-04-30T06:44:16.296ZTRACtest_utilization (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal
119902025-04-30T06:44:16.298ZTRACtest_utilization (dns): buffer buffer = Length: 67 (0x43) bytes\n0000: c9 a0 01 00 00 01 00 00 00 00 00 01 04 5f 6d 67 ............._mg\n0010: 73 04 5f 74 63 70 0d 63 6f 6e 74 72 6f 6c 2d 70 s._tcp.control-p\n0020: 6c 61 6e 65 05 6f 78 69 64 65 08 69 6e 74 65 72 lane.oxide.inter\n0030: 6e 61 6c 00 00 21 00 01 00 00 29 04 d0 00 00 00 nal..!....).....\n0040: 00 00 00 ... kind = dns peer_addr = [::1]:63874 req_id = 4194ee76-eacc-44e0-9850-13730163cc9f
119912025-04-30T06:44:16.298ZDEBGtest_utilization (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 51616,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_mgs._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_mgs._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 4,\n 95,\n 109,\n 103,\n 115,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:63874 req_id = 4194ee76-eacc-44e0-9850-13730163cc9f
119922025-04-30T06:44:16.298ZDEBGtest_utilization (store): query key key = _mgs._tcp
119932025-04-30T06:44:16.298ZDEBGtest_utilization (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
119942025-04-30T06:44:16.298ZDEBGtest_utilization (dns): dns response additional_records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal"), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] kind = dns peer_addr = [::1]:63874 query = LowerQuery { name: LowerName(Name("_mgs._tcp.control-plane.oxide.internal.")), original: Query { name: Name("_mgs._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN } } records = [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 48759, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal") })) }] req_id = 4194ee76-eacc-44e0-9850-13730163cc9f
119952025-04-30T06:44:16.299ZDEBGtest_utilization (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgs._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 48759, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }], valid_until: Instant { tv_sec: 4321, tv_nsec: 97924661 } })
119962025-04-30T06:44:16.299ZTRACtest_utilization (DnsResolver): lookup_service_targets: looking up SRV target
119972025-04-30T06:44:16.299ZTRACtest_utilization (dns): buffer buffer = Length: 108 (0x6c) bytes\n0000: 02 06 01 00 00 01 00 00 00 00 00 01 2d 64 65 6e ............-den\n0010: 64 72 69 74 65 2d 62 36 64 36 35 33 34 31 2d 31 drite-b6d65341-1\n0020: 36 37 63 2d 34 31 64 66 2d 39 62 35 63 2d 34 31 67c-41df-9b5c-41\n0030: 63 64 65 64 39 39 63 32 32 39 04 68 6f 73 74 0d cded99c229.host.\n0040: 63 6f 6e 74 72 6f 6c 2d 70 6c 61 6e 65 05 6f 78 control-plane.ox\n0050: 69 64 65 08 69 6e 74 65 72 6e 61 6c 00 00 1c 00 ide.internal....\n0060: 01 00 00 29 04 d0 00 00 00 00 00 00 ...)........ kind = dns peer_addr = [::1]:61751 req_id = 62ec8bd8-68d6-4e8c-8729-610503fcded8
119982025-04-30T06:44:16.299ZDEBGtest_utilization (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 518,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n query_type: AAAA,\n query_class: IN,\n },\n },\n original: [\n 45,\n 100,\n 101,\n 110,\n 100,\n 114,\n 105,\n 116,\n 101,\n 45,\n 98,\n 54,\n 100,\n 54,\n 53,\n 51,\n 52,\n 49,\n 45,\n 49,\n 54,\n 55,\n 99,\n 45,\n 52,\n 49,\n 100,\n 102,\n 45,\n 57,\n 98,\n 53,\n 99,\n 45,\n 52,\n 49,\n 99,\n 100,\n 101,\n 100,\n 57,\n 57,\n 99,\n 50,\n 50,\n 57,\n 4,\n 104,\n 111,\n 115,\n 116,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 28,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:61751 req_id = 62ec8bd8-68d6-4e8c-8729-610503fcded8
119992025-04-30T06:44:16.299ZDEBGtest_utilization (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
120002025-04-30T06:44:16.299ZDEBGtest_utilization (dns): dns response additional_records = [] kind = dns peer_addr = [::1]:61751 query = LowerQuery { name: LowerName(Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.")), original: Query { name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), query_type: AAAA, query_class: IN } } records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] req_id = 62ec8bd8-68d6-4e8c-8729-610503fcded8
120012025-04-30T06:44:16.361ZTRACtest_utilization (DnsResolver): lookup_all_socket_v6 srv dns_name = _clickhouse-admin-keeper._tcp.control-plane.oxide.internal
120022025-04-30T06:44:16.362ZTRACtest_utilization (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory
120032025-04-30T06:44:16.362ZTRACtest_utilization (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
120042025-04-30T06:44:16.362ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
120052025-04-30T06:44:16.362ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
120062025-04-30T06:44:16.363ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
120072025-04-30T06:44:16.363ZDEBGtest_utilization (results-sink): inserting 1 samples into database collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
120082025-04-30T06:44:16.363ZDEBGtest_utilization (results-sink-cluster): inserting 1 samples into database collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
120092025-04-30T06:44:16.364ZDEBGtest_utilization (clickhouse-client): unrolling 1 total samples collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 4d1485f0-110e-4161-9a33-c4610d68be5d
120102025-04-30T06:44:16.364ZTRACtest_utilization (clickhouse-client): inserting data collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 4d1485f0-110e-4161-9a33-c4610d68be5d n_columns = 4 n_rows = 2 sql = INSERT INTO oximeter.fields_string FORMAT Native
120112025-04-30T06:44:16.365ZTRACtest_utilization (dns): buffer buffer = Length: 87 (0x57) bytes\n0000: 57 4d 01 00 00 01 00 00 00 00 00 01 18 5f 63 6c WM..........._cl\n0010: 69 63 6b 68 6f 75 73 65 2d 61 64 6d 69 6e 2d 6b ickhouse-admin-k\n0020: 65 65 70 65 72 04 5f 74 63 70 0d 63 6f 6e 74 72 eeper._tcp.contr\n0030: 6f 6c 2d 70 6c 61 6e 65 05 6f 78 69 64 65 08 69 ol-plane.oxide.i\n0040: 6e 74 65 72 6e 61 6c 00 00 21 00 01 00 00 29 04 nternal..!....).\n0050: d0 00 00 00 00 00 00 ....... kind = dns peer_addr = [::1]:51465 req_id = f85a8b18-324f-4ac9-a2ab-d833da75881e
120122025-04-30T06:44:16.365ZDEBGtest_utilization (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 22349,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 24,\n 95,\n 99,\n 108,\n 105,\n 99,\n 107,\n 104,\n 111,\n 117,\n 115,\n 101,\n 45,\n 97,\n 100,\n 109,\n 105,\n 110,\n 45,\n 107,\n 101,\n 101,\n 112,\n 101,\n 114,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:51465 req_id = f85a8b18-324f-4ac9-a2ab-d833da75881e
120132025-04-30T06:44:16.365ZDEBGtest_utilization (store): query key key = _clickhouse-admin-keeper._tcp
120142025-04-30T06:44:16.365ZERROtest_utilization (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 22349, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 1, }, query: WireQuery { query: LowerQuery { name: LowerName( Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), ), original: Query { name: Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN, }, }, original: [ 24, 95, 99, 108, 105, 99, 107, 104, 111, 117, 115, 101, 45, 97, 100, 109, 105, 110, 45, 107, 101, 101, 112, 101, 114, 4, 95, 116, 99, 112, 13, 99, 111, 110, 116, 114, 111, 108, 45, 112, 108, 97, 110, 101, 5, 111, 120, 105, 100, 101, 8, 105, 110, 116, 101, 114, 110, 97, 108, 0, 0, 33, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: Some( Edns { rcode_high: 0, version: 0, dnssec_ok: false, max_payload: 1232, options: OPT { options: {}, }, }, ), } NXDOMAIN: no records found for name: "_clickhouse-admin-keeper._tcp.control-plane.oxide.internal." kind = dns peer_addr = [::1]:51465 req_id = f85a8b18-324f-4ac9-a2ab-d833da75881e
120152025-04-30T06:44:16.365ZDEBGtest_utilization (BackgroundTasks): begin collection background_task = inventory_collection
120162025-04-30T06:44:16.365ZDEBGtest_utilization (BackgroundTasks): begin collection from MGS background_task = inventory_collection mgs_url = http://[::1]:48759
120172025-04-30T06:44:16.365ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/ignition
120182025-04-30T06:44:16.366ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
120192025-04-30T06:44:16.372ZDEBGtest_utilization (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory result = Ok(())
120202025-04-30T06:44:16.372ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
120212025-04-30T06:44:16.372ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
120222025-04-30T06:44:16.373ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
120232025-04-30T06:44:16.374ZDEBGtest_utilization (clickhouse-client): inserted rows into table collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 4d1485f0-110e-4161-9a33-c4610d68be5d n_rows = 2 table_name = fields_string
120242025-04-30T06:44:16.374ZTRACtest_utilization (clickhouse-client): inserting data collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 4d1485f0-110e-4161-9a33-c4610d68be5d n_columns = 4 n_rows = 1 sql = INSERT INTO oximeter.measurements_i64 FORMAT Native
120252025-04-30T06:44:16.375ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/ignition", status: 200, headers: {"content-type": "application/json", "x-request-id": "7cb1477e-a7c9-48fb-ab4b-1209329335a1", "content-length": "835", "date": "Wed, 30 Apr 2025 06:44:15 GMT"} })
120262025-04-30T06:44:16.375ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0
120272025-04-30T06:44:16.376ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
120282025-04-30T06:44:16.376ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
120292025-04-30T06:44:16.376ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
120302025-04-30T06:44:16.378ZDEBGtest_utilization (clickhouse-client): inserted rows into table collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 4d1485f0-110e-4161-9a33-c4610d68be5d n_rows = 1 table_name = measurements_i64
120312025-04-30T06:44:16.378ZTRACtest_utilization (results-sink): successfully inserted samples collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
120322025-04-30T06:44:16.379ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "8fbe035d-5aab-4639-832e-4980b8f9feb6", "content-length": "734", "date": "Wed, 30 Apr 2025 06:44:15 GMT"} })
120332025-04-30T06:44:16.379ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0/component/sp/caboose?firmware_slot=0
120342025-04-30T06:44:16.381ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "e16d1dbd-e96d-4aa1-8fdd-27a23b663e6f", "content-length": "111", "date": "Wed, 30 Apr 2025 06:44:15 GMT"} })
120352025-04-30T06:44:16.381ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0/component/sp/caboose?firmware_slot=1
120362025-04-30T06:44:16.384ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "f5eed58b-5799-49bd-bdef-e9433e62af33", "content-length": "111", "date": "Wed, 30 Apr 2025 06:44:15 GMT"} })
120372025-04-30T06:44:16.384ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0/component/rot/caboose?firmware_slot=0
120382025-04-30T06:44:16.389ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "41b7297d-4a17-497b-811a-d744a8aa9dc7", "content-length": "170", "date": "Wed, 30 Apr 2025 06:44:15 GMT"} })
120392025-04-30T06:44:16.389ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0/component/rot/caboose?firmware_slot=1
120402025-04-30T06:44:16.392ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "d6772f7c-f421-4506-af83-b33f797638b3", "content-length": "170", "date": "Wed, 30 Apr 2025 06:44:15 GMT"} })
120412025-04-30T06:44:16.392ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0/component/stage0/caboose?firmware_slot=0
120422025-04-30T06:44:16.395ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "405c5fd4-c022-49be-8a85-15c84290e092", "content-length": "179", "date": "Wed, 30 Apr 2025 06:44:15 GMT"} })
120432025-04-30T06:44:16.395ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0/component/stage0/caboose?firmware_slot=1
120442025-04-30T06:44:16.397ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "5d6d90cb-4002-4a2f-86b2-0d62d5bad4dd", "content-length": "179", "date": "Wed, 30 Apr 2025 06:44:15 GMT"} })
120452025-04-30T06:44:16.397ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0/component/rot/cmpa
120462025-04-30T06:44:16.398ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "79662454-2671-4528-893b-e0c3277d9573", "content-length": "702", "date": "Wed, 30 Apr 2025 06:44:15 GMT"} })
120472025-04-30T06:44:16.398ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0/component/rot/cfpa
120482025-04-30T06:44:16.399ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "8a6ae9ad-b071-4612-a8f8-3527715c2a2c", "content-length": "727", "date": "Wed, 30 Apr 2025 06:44:15 GMT"} })
120492025-04-30T06:44:16.399ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0/component/rot/cfpa
120502025-04-30T06:44:16.400ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "af4bc406-bebb-41f1-bb5b-987c6886c4da", "content-length": "729", "date": "Wed, 30 Apr 2025 06:44:15 GMT"} })
120512025-04-30T06:44:16.401ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0/component/rot/cfpa
120522025-04-30T06:44:16.402ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "8e17c803-1220-40bd-a842-90a6dfdecb64", "content-length": "728", "date": "Wed, 30 Apr 2025 06:44:15 GMT"} })
120532025-04-30T06:44:16.402ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1
120542025-04-30T06:44:16.407ZDEBGtest_utilization: attempt to fetch latest inventory collection result = Ok("found")
120552025-04-30T06:44:16.409ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1", status: 200, headers: {"content-type": "application/json", "x-request-id": "0ac54044-8234-42b2-aa05-838c56dea9d0", "content-length": "734", "date": "Wed, 30 Apr 2025 06:44:15 GMT"} })
120562025-04-30T06:44:16.409ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1/component/sp/caboose?firmware_slot=0
120572025-04-30T06:44:16.412ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "6df4ac2e-881b-4244-b862-993144209bae", "content-length": "111", "date": "Wed, 30 Apr 2025 06:44:15 GMT"} })
120582025-04-30T06:44:16.412ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1/component/sp/caboose?firmware_slot=1
120592025-04-30T06:44:16.415ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "c62f8d5f-9165-4fb7-a874-a4ee0f64a858", "content-length": "111", "date": "Wed, 30 Apr 2025 06:44:15 GMT"} })
120602025-04-30T06:44:16.415ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1/component/rot/caboose?firmware_slot=0
120612025-04-30T06:44:16.419ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "fb93990d-64dd-4fbd-9451-9a37caafe907", "content-length": "170", "date": "Wed, 30 Apr 2025 06:44:15 GMT"} })
120622025-04-30T06:44:16.419ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1/component/rot/caboose?firmware_slot=1
120632025-04-30T06:44:16.421ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "e083696c-f995-4782-b85b-9e38607d940e", "content-length": "170", "date": "Wed, 30 Apr 2025 06:44:15 GMT"} })
120642025-04-30T06:44:16.421ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1/component/stage0/caboose?firmware_slot=0
120652025-04-30T06:44:16.424ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "3030f196-8417-4499-a56e-6d29a4ddddcc", "content-length": "179", "date": "Wed, 30 Apr 2025 06:44:15 GMT"} })
120662025-04-30T06:44:16.424ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1/component/stage0/caboose?firmware_slot=1
120672025-04-30T06:44:16.428ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "cc628941-4739-4f46-9bbd-b72903453512", "content-length": "179", "date": "Wed, 30 Apr 2025 06:44:15 GMT"} })
120682025-04-30T06:44:16.428ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1/component/rot/cmpa
120692025-04-30T06:44:16.432ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "6ec532b4-90b7-483e-8a5a-1d50ad20c7cf", "content-length": "702", "date": "Wed, 30 Apr 2025 06:44:15 GMT"} })
120702025-04-30T06:44:16.433ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1/component/rot/cfpa
120712025-04-30T06:44:16.435ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "05639a15-1f08-4775-890f-bec73e946acf", "content-length": "727", "date": "Wed, 30 Apr 2025 06:44:15 GMT"} })
120722025-04-30T06:44:16.435ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1/component/rot/cfpa
120732025-04-30T06:44:16.440ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "8af8d3d9-57f6-40a4-a0bf-c9702268208e", "content-length": "729", "date": "Wed, 30 Apr 2025 06:44:15 GMT"} })
120742025-04-30T06:44:16.440ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1/component/rot/cfpa
120752025-04-30T06:44:16.442ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "337d93af-2529-441e-9eb6-c751e60a2519", "content-length": "728", "date": "Wed, 30 Apr 2025 06:44:15 GMT"} })
120762025-04-30T06:44:16.442ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0
120772025-04-30T06:44:16.444ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "50053e72-2535-4a0a-82cb-472b727ccecd", "content-length": "723", "date": "Wed, 30 Apr 2025 06:44:15 GMT"} })
120782025-04-30T06:44:16.444ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0/component/sp/caboose?firmware_slot=0
120792025-04-30T06:44:16.446ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "c2a74ff8-952d-461c-8805-a21315e28ba5", "content-length": "109", "date": "Wed, 30 Apr 2025 06:44:15 GMT"} })
120802025-04-30T06:44:16.446ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0/component/sp/caboose?firmware_slot=1
120812025-04-30T06:44:16.448ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "815fb706-af74-4846-b825-1d5390e4e653", "content-length": "109", "date": "Wed, 30 Apr 2025 06:44:15 GMT"} })
120822025-04-30T06:44:16.448ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0/component/rot/caboose?firmware_slot=0
120832025-04-30T06:44:16.450ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "19976de4-19a1-4653-bd6e-6af5c06ad8ec", "content-length": "169", "date": "Wed, 30 Apr 2025 06:44:15 GMT"} })
120842025-04-30T06:44:16.450ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0/component/rot/caboose?firmware_slot=1
120852025-04-30T06:44:16.454ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "420ede7c-a7a6-49b9-9e6a-e12d70d15b74", "content-length": "169", "date": "Wed, 30 Apr 2025 06:44:15 GMT"} })
120862025-04-30T06:44:16.454ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0/component/stage0/caboose?firmware_slot=0
120872025-04-30T06:44:16.457ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "293155b8-797f-4c3e-852f-8d97716bc619", "content-length": "178", "date": "Wed, 30 Apr 2025 06:44:15 GMT"} })
120882025-04-30T06:44:16.457ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0/component/stage0/caboose?firmware_slot=1
120892025-04-30T06:44:16.457ZTRACtest_utilization (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory
120902025-04-30T06:44:16.457ZTRACtest_utilization (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
120912025-04-30T06:44:16.458ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
120922025-04-30T06:44:16.458ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
120932025-04-30T06:44:16.459ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
120942025-04-30T06:44:16.461ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "ec1f5549-99ac-4129-992a-332ba134db1b", "content-length": "178", "date": "Wed, 30 Apr 2025 06:44:15 GMT"} })
120952025-04-30T06:44:16.461ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0/component/rot/cmpa
120962025-04-30T06:44:16.462ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "f3d8c4bf-5614-4aaf-9b1e-f3f76f8c4cb2", "content-length": "702", "date": "Wed, 30 Apr 2025 06:44:15 GMT"} })
120972025-04-30T06:44:16.462ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0/component/rot/cfpa
120982025-04-30T06:44:16.463ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
120992025-04-30T06:44:16.473ZDEBGtest_utilization (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory result = Ok(())
121002025-04-30T06:44:16.473ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
121012025-04-30T06:44:16.473ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
121022025-04-30T06:44:16.473ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
121032025-04-30T06:44:16.477ZWARNtest_utilization (results-sink-cluster): failed to insert some results into metric DB collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 error = Failed to check out connection to database: No backends found for this service file = oximeter/collector/src/results_sink.rs:92
121042025-04-30T06:44:16.478ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
121052025-04-30T06:44:16.478ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
121062025-04-30T06:44:16.479ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
121072025-04-30T06:44:16.480ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "9a7ac519-194a-413e-864f-8328a1d023fa", "content-length": "727", "date": "Wed, 30 Apr 2025 06:44:15 GMT"} })
121082025-04-30T06:44:16.480ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0/component/rot/cfpa
121092025-04-30T06:44:16.482ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "a9f515de-5ef6-4622-9425-a22592fd2794", "content-length": "729", "date": "Wed, 30 Apr 2025 06:44:15 GMT"} })
121102025-04-30T06:44:16.482ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0/component/rot/cfpa
121112025-04-30T06:44:16.485ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "ef33c35d-c86e-473f-9e36-052f6ecb2d13", "content-length": "728", "date": "Wed, 30 Apr 2025 06:44:15 GMT"} })
121122025-04-30T06:44:16.486ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1
121132025-04-30T06:44:16.490ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1", status: 200, headers: {"content-type": "application/json", "x-request-id": "d67f09d3-9c68-41ba-aad7-5addc01ffffa", "content-length": "723", "date": "Wed, 30 Apr 2025 06:44:15 GMT"} })
121142025-04-30T06:44:16.492ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1/component/sp/caboose?firmware_slot=0
121152025-04-30T06:44:16.506ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "af4799b0-fbd0-4838-9f31-4db9927b021a", "content-length": "109", "date": "Wed, 30 Apr 2025 06:44:15 GMT"} })
121162025-04-30T06:44:16.506ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1/component/sp/caboose?firmware_slot=1
121172025-04-30T06:44:16.516ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "31d3895d-68b4-4a18-a182-d3a8af267929", "content-length": "109", "date": "Wed, 30 Apr 2025 06:44:15 GMT"} })
121182025-04-30T06:44:16.516ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1/component/rot/caboose?firmware_slot=0
121192025-04-30T06:44:16.522ZDEBGtest_utilization: attempt to fetch latest inventory collection result = Ok("found")
121202025-04-30T06:44:16.522ZTRACtest_utilization (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal
121212025-04-30T06:44:16.546ZTRACtest_utilization (dns): buffer buffer = Length: 72 (0x48) bytes\n0000: fc d3 01 00 00 01 00 00 00 00 00 01 09 5f 64 65 ............._de\n0010: 6e 64 72 69 74 65 04 5f 74 63 70 0d 63 6f 6e 74 ndrite._tcp.cont\n0020: 72 6f 6c 2d 70 6c 61 6e 65 05 6f 78 69 64 65 08 rol-plane.oxide.\n0030: 69 6e 74 65 72 6e 61 6c 00 00 21 00 01 00 00 29 internal..!....)\n0040: 04 d0 00 00 00 00 00 00 ........ kind = dns peer_addr = [::1]:61414 req_id = 1a8b40be-c335-4fa8-b1db-c64d3342e37a
121222025-04-30T06:44:16.546ZDEBGtest_utilization (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 64723,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_dendrite._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_dendrite._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 9,\n 95,\n 100,\n 101,\n 110,\n 100,\n 114,\n 105,\n 116,\n 101,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:61414 req_id = 1a8b40be-c335-4fa8-b1db-c64d3342e37a
121232025-04-30T06:44:16.547ZDEBGtest_utilization (store): query key key = _dendrite._tcp
121242025-04-30T06:44:16.547ZDEBGtest_utilization (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
121252025-04-30T06:44:16.547ZDEBGtest_utilization (dns): dns response additional_records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal"), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] kind = dns peer_addr = [::1]:61414 query = LowerQuery { name: LowerName(Name("_dendrite._tcp.control-plane.oxide.internal.")), original: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN } } records = [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 47974, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal") })) }] req_id = 1a8b40be-c335-4fa8-b1db-c64d3342e37a
121262025-04-30T06:44:16.547ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "50edb9e8-da4a-485b-9c5c-680df2227328", "content-length": "169", "date": "Wed, 30 Apr 2025 06:44:15 GMT"} })
121272025-04-30T06:44:16.548ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1/component/rot/caboose?firmware_slot=1
121282025-04-30T06:44:16.548ZDEBGtest_utilization (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 47974, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }], valid_until: Instant { tv_sec: 4321, tv_nsec: 347859861 } })
121292025-04-30T06:44:16.549ZTRACtest_utilization (DnsResolver): lookup_service_targets: looking up SRV target
121302025-04-30T06:44:16.549ZTRACtest_utilization (dns): buffer buffer = Length: 108 (0x6c) bytes\n0000: 89 0b 01 00 00 01 00 00 00 00 00 01 2d 64 65 6e ............-den\n0010: 64 72 69 74 65 2d 62 36 64 36 35 33 34 31 2d 31 drite-b6d65341-1\n0020: 36 37 63 2d 34 31 64 66 2d 39 62 35 63 2d 34 31 67c-41df-9b5c-41\n0030: 63 64 65 64 39 39 63 32 32 39 04 68 6f 73 74 0d cded99c229.host.\n0040: 63 6f 6e 74 72 6f 6c 2d 70 6c 61 6e 65 05 6f 78 control-plane.ox\n0050: 69 64 65 08 69 6e 74 65 72 6e 61 6c 00 00 1c 00 ide.internal....\n0060: 01 00 00 29 04 d0 00 00 00 00 00 00 ...)........ kind = dns peer_addr = [::1]:49331 req_id = bebadb2a-bade-4f4a-8d1d-310b1613eafd
121312025-04-30T06:44:16.549ZDEBGtest_utilization (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 35083,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n query_type: AAAA,\n query_class: IN,\n },\n },\n original: [\n 45,\n 100,\n 101,\n 110,\n 100,\n 114,\n 105,\n 116,\n 101,\n 45,\n 98,\n 54,\n 100,\n 54,\n 53,\n 51,\n 52,\n 49,\n 45,\n 49,\n 54,\n 55,\n 99,\n 45,\n 52,\n 49,\n 100,\n 102,\n 45,\n 57,\n 98,\n 53,\n 99,\n 45,\n 52,\n 49,\n 99,\n 100,\n 101,\n 100,\n 57,\n 57,\n 99,\n 50,\n 50,\n 57,\n 4,\n 104,\n 111,\n 115,\n 116,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 28,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:49331 req_id = bebadb2a-bade-4f4a-8d1d-310b1613eafd
121322025-04-30T06:44:16.549ZDEBGtest_utilization (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
121332025-04-30T06:44:16.549ZDEBGtest_utilization (dns): dns response additional_records = [] kind = dns peer_addr = [::1]:49331 query = LowerQuery { name: LowerName(Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.")), original: Query { name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), query_type: AAAA, query_class: IN } } records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] req_id = bebadb2a-bade-4f4a-8d1d-310b1613eafd
121342025-04-30T06:44:16.550ZINFOtest_utilization (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_v4_garbage_collector
121352025-04-30T06:44:16.617ZINFOtest_utilization (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_v4_garbage_collector zone_address = ::1
121362025-04-30T06:44:16.617ZDEBGtest_utilization (MgsClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[::1]:12225/local/switch-id
121372025-04-30T06:44:16.618ZTRACtest_utilization (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory
121382025-04-30T06:44:16.618ZTRACtest_utilization (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
121392025-04-30T06:44:16.618ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
121402025-04-30T06:44:16.618ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
121412025-04-30T06:44:16.621ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
121422025-04-30T06:44:16.622ZDEBGtest_utilization (MgsClient): client response background_task = nat_v4_garbage_collector result = Err(reqwest::Error { kind: Request, url: "http://[::1]:12225/local/switch-id", source: hyper_util::client::legacy::Error(Connect, ConnectError("tcp connect error", Os { code: 146, kind: ConnectionRefused, message: "Connection refused" })) })
121432025-04-30T06:44:16.622ZWARNtest_utilization (BackgroundTasks): failed to identify switch slot for dendrite background_task = nat_v4_garbage_collector reason = Communication Error: error sending request for url (http://[::1]:12225/local/switch-id): client error (Connect): tcp connect error: Connection refused (os error 146): Connection refused (os error 146) zone_address = ::1
121442025-04-30T06:44:16.626ZWARNtest_utilization (BackgroundTasks): Failed to map switch zone addr: Communication Error: error sending request for url (http://[::1]:12225/local/switch-id), retrying background_task = nat_v4_garbage_collector
121452025-04-30T06:44:16.626ZTRACtest_utilization (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal
121462025-04-30T06:44:16.629ZTRACtest_utilization (dns): buffer buffer = Length: 72 (0x48) bytes\n0000: b9 0d 01 00 00 01 00 00 00 00 00 01 09 5f 64 65 ............._de\n0010: 6e 64 72 69 74 65 04 5f 74 63 70 0d 63 6f 6e 74 ndrite._tcp.cont\n0020: 72 6f 6c 2d 70 6c 61 6e 65 05 6f 78 69 64 65 08 rol-plane.oxide.\n0030: 69 6e 74 65 72 6e 61 6c 00 00 21 00 01 00 00 29 internal..!....)\n0040: 04 d0 00 00 00 00 00 00 ........ kind = dns peer_addr = [::1]:50965 req_id = fd03a5c8-04b4-4eec-9e16-b2f6e90de38c
121472025-04-30T06:44:16.629ZDEBGtest_utilization (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 47373,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_dendrite._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_dendrite._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 9,\n 95,\n 100,\n 101,\n 110,\n 100,\n 114,\n 105,\n 116,\n 101,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:50965 req_id = fd03a5c8-04b4-4eec-9e16-b2f6e90de38c
121482025-04-30T06:44:16.630ZDEBGtest_utilization (store): query key key = _dendrite._tcp
121492025-04-30T06:44:16.630ZDEBGtest_utilization (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
121502025-04-30T06:44:16.630ZDEBGtest_utilization (dns): dns response additional_records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal"), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] kind = dns peer_addr = [::1]:50965 query = LowerQuery { name: LowerName(Name("_dendrite._tcp.control-plane.oxide.internal.")), original: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN } } records = [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 47974, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal") })) }] req_id = fd03a5c8-04b4-4eec-9e16-b2f6e90de38c
121512025-04-30T06:44:16.632ZDEBGtest_utilization (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 47974, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }], valid_until: Instant { tv_sec: 4321, tv_nsec: 430518383 } })
121522025-04-30T06:44:16.632ZTRACtest_utilization (DnsResolver): lookup_service_targets: looking up SRV target
121532025-04-30T06:44:16.632ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
121542025-04-30T06:44:16.641ZDEBGtest_utilization (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory result = Ok(())
121552025-04-30T06:44:16.641ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
121562025-04-30T06:44:16.642ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
121572025-04-30T06:44:16.643ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
121582025-04-30T06:44:16.652ZTRACtest_utilization (dns): buffer buffer = Length: 108 (0x6c) bytes\n0000: 3e be 01 00 00 01 00 00 00 00 00 01 2d 64 65 6e >...........-den\n0010: 64 72 69 74 65 2d 62 36 64 36 35 33 34 31 2d 31 drite-b6d65341-1\n0020: 36 37 63 2d 34 31 64 66 2d 39 62 35 63 2d 34 31 67c-41df-9b5c-41\n0030: 63 64 65 64 39 39 63 32 32 39 04 68 6f 73 74 0d cded99c229.host.\n0040: 63 6f 6e 74 72 6f 6c 2d 70 6c 61 6e 65 05 6f 78 control-plane.ox\n0050: 69 64 65 08 69 6e 74 65 72 6e 61 6c 00 00 1c 00 ide.internal....\n0060: 01 00 00 29 04 d0 00 00 00 00 00 00 ...)........ kind = dns peer_addr = [::1]:59087 req_id = 64ee077c-29ba-4cf3-b989-3d9123a44fbe
121592025-04-30T06:44:16.652ZDEBGtest_utilization (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 16062,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n query_type: AAAA,\n query_class: IN,\n },\n },\n original: [\n 45,\n 100,\n 101,\n 110,\n 100,\n 114,\n 105,\n 116,\n 101,\n 45,\n 98,\n 54,\n 100,\n 54,\n 53,\n 51,\n 52,\n 49,\n 45,\n 49,\n 54,\n 55,\n 99,\n 45,\n 52,\n 49,\n 100,\n 102,\n 45,\n 57,\n 98,\n 53,\n 99,\n 45,\n 52,\n 49,\n 99,\n 100,\n 101,\n 100,\n 57,\n 57,\n 99,\n 50,\n 50,\n 57,\n 4,\n 104,\n 111,\n 115,\n 116,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 28,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:59087 req_id = 64ee077c-29ba-4cf3-b989-3d9123a44fbe
121602025-04-30T06:44:16.652ZDEBGtest_utilization (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
121612025-04-30T06:44:16.652ZDEBGtest_utilization (dns): dns response additional_records = [] kind = dns peer_addr = [::1]:59087 query = LowerQuery { name: LowerName(Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.")), original: Query { name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), query_type: AAAA, query_class: IN } } records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] req_id = 64ee077c-29ba-4cf3-b989-3d9123a44fbe
121622025-04-30T06:44:16.653ZINFOtest_utilization (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager
121632025-04-30T06:44:16.708ZINFOtest_utilization (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager zone_address = ::1
121642025-04-30T06:44:16.708ZDEBGtest_utilization (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[::1]:12225/local/switch-id
121652025-04-30T06:44:16.709ZDEBGtest_utilization (MgsClient): client response background_task = bfd_manager result = Err(reqwest::Error { kind: Request, url: "http://[::1]:12225/local/switch-id", source: hyper_util::client::legacy::Error(Connect, ConnectError("tcp connect error", Os { code: 146, kind: ConnectionRefused, message: "Connection refused" })) })
121662025-04-30T06:44:16.709ZWARNtest_utilization (BackgroundTasks): failed to identify switch slot for dendrite background_task = bfd_manager reason = Communication Error: error sending request for url (http://[::1]:12225/local/switch-id): client error (Connect): tcp connect error: Connection refused (os error 146): Connection refused (os error 146) zone_address = ::1
121672025-04-30T06:44:16.710ZWARNtest_utilization (BackgroundTasks): Failed to map switch zone addr: Communication Error: error sending request for url (http://[::1]:12225/local/switch-id), retrying background_task = bfd_manager
121682025-04-30T06:44:16.711ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
121692025-04-30T06:44:16.712ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
121702025-04-30T06:44:16.712ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
121712025-04-30T06:44:16.728ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "79fe52e2-5c79-4caf-a48b-14698ce48d98", "content-length": "169", "date": "Wed, 30 Apr 2025 06:44:15 GMT"} })
121722025-04-30T06:44:16.729ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1/component/stage0/caboose?firmware_slot=0
121732025-04-30T06:44:16.756ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "b1800036-e920-468c-aa08-8c69fe4d6260", "content-length": "178", "date": "Wed, 30 Apr 2025 06:44:15 GMT"} })
121742025-04-30T06:44:16.756ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1/component/stage0/caboose?firmware_slot=1
121752025-04-30T06:44:16.760ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "db760f14-81ca-4147-9cc7-a38ec0df0fcd", "content-length": "178", "date": "Wed, 30 Apr 2025 06:44:15 GMT"} })
121762025-04-30T06:44:16.760ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1/component/rot/cmpa
121772025-04-30T06:44:16.762ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "42008129-ffbe-4268-8dd1-6102228d1d39", "content-length": "702", "date": "Wed, 30 Apr 2025 06:44:15 GMT"} })
121782025-04-30T06:44:16.763ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1/component/rot/cfpa
121792025-04-30T06:44:16.764ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "c1d8544b-9c3d-4bd0-8b5a-2de6f6d8032f", "content-length": "727", "date": "Wed, 30 Apr 2025 06:44:15 GMT"} })
121802025-04-30T06:44:16.764ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1/component/rot/cfpa
121812025-04-30T06:44:16.770ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "5a2f09d1-56d9-4626-b48f-da49bed658ae", "content-length": "729", "date": "Wed, 30 Apr 2025 06:44:15 GMT"} })
121822025-04-30T06:44:16.770ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1/component/rot/cfpa
121832025-04-30T06:44:16.774ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "eb9da739-0cdf-4c81-bdbf-e67ecc52f73d", "content-length": "728", "date": "Wed, 30 Apr 2025 06:44:15 GMT"} })
121842025-04-30T06:44:16.774ZTRACtest_utilization (BackgroundTasks): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet
121852025-04-30T06:44:16.774ZTRACtest_utilization (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
121862025-04-30T06:44:16.774ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
121872025-04-30T06:44:16.774ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
121882025-04-30T06:44:16.775ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
121892025-04-30T06:44:16.776ZDEBGtest_utilization: attempt to fetch latest inventory collection result = Ok("found")
121902025-04-30T06:44:16.781ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
121912025-04-30T06:44:16.786ZDEBGtest_utilization (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet result = Ok(())
121922025-04-30T06:44:16.786ZTRACtest_utilization (BackgroundTasks): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet
121932025-04-30T06:44:16.786ZTRACtest_utilization (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
121942025-04-30T06:44:16.786ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
121952025-04-30T06:44:16.786ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
121962025-04-30T06:44:16.788ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
121972025-04-30T06:44:16.792ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
121982025-04-30T06:44:16.796ZDEBGtest_utilization (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet result = Ok(())
121992025-04-30T06:44:16.796ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
122002025-04-30T06:44:16.796ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
122012025-04-30T06:44:16.796ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
122022025-04-30T06:44:16.905ZDEBGtest_utilization (BackgroundTasks): begin collection from Sled Agent background_task = inventory_collection sled_agent_url = http://[::1]:39365
122032025-04-30T06:44:16.905ZDEBGtest_utilization (BackgroundTasks): client request SledAgent = http://[::1]:39365 background_task = inventory_collection body = None method = GET uri = http://[::1]:39365/inventory
122042025-04-30T06:44:16.906ZTRACtest_utilization (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory
122052025-04-30T06:44:16.906ZTRACtest_utilization (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
122062025-04-30T06:44:16.906ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
122072025-04-30T06:44:16.906ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
122082025-04-30T06:44:16.908ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
122092025-04-30T06:44:16.908ZINFOtest_utilization (dropshot): accepted connection local_addr = [::1]:39365 remote_addr = [::1]:36456 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
122102025-04-30T06:44:16.908ZTRACtest_utilization (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal
122112025-04-30T06:44:16.912ZTRACtest_utilization (dropshot): incoming request local_addr = [::1]:39365 method = GET remote_addr = [::1]:36456 req_id = 71a3e7fe-005c-4c16-b473-83b3ef8c6528 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /inventory
122122025-04-30T06:44:16.912ZINFOtest_utilization (dropshot): request completed latency_us = 655 local_addr = [::1]:39365 method = GET remote_addr = [::1]:36456 req_id = 71a3e7fe-005c-4c16-b473-83b3ef8c6528 response_code = 200 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /inventory
122132025-04-30T06:44:16.913ZTRACtest_utilization (dns): buffer buffer = Length: 72 (0x48) bytes\n0000: 83 7a 01 00 00 01 00 00 00 00 00 01 09 5f 64 65 .z..........._de\n0010: 6e 64 72 69 74 65 04 5f 74 63 70 0d 63 6f 6e 74 ndrite._tcp.cont\n0020: 72 6f 6c 2d 70 6c 61 6e 65 05 6f 78 69 64 65 08 rol-plane.oxide.\n0030: 69 6e 74 65 72 6e 61 6c 00 00 21 00 01 00 00 29 internal..!....)\n0040: 04 d0 00 00 00 00 00 00 ........ kind = dns peer_addr = [::1]:57140 req_id = de07f770-99ad-4649-9784-25081e967a56
122142025-04-30T06:44:16.913ZDEBGtest_utilization (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 33658,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_dendrite._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_dendrite._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 9,\n 95,\n 100,\n 101,\n 110,\n 100,\n 114,\n 105,\n 116,\n 101,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:57140 req_id = de07f770-99ad-4649-9784-25081e967a56
122152025-04-30T06:44:16.913ZDEBGtest_utilization (store): query key key = _dendrite._tcp
122162025-04-30T06:44:16.913ZDEBGtest_utilization (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
122172025-04-30T06:44:16.913ZDEBGtest_utilization (dns): dns response additional_records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal"), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] kind = dns peer_addr = [::1]:57140 query = LowerQuery { name: LowerName(Name("_dendrite._tcp.control-plane.oxide.internal.")), original: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN } } records = [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 47974, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal") })) }] req_id = de07f770-99ad-4649-9784-25081e967a56
122182025-04-30T06:44:16.914ZDEBGtest_utilization (BackgroundTasks): client response SledAgent = http://[::1]:39365 background_task = inventory_collection result = Ok(Response { url: "http://[::1]:39365/inventory", status: 200, headers: {"content-type": "application/json", "x-request-id": "71a3e7fe-005c-4c16-b473-83b3ef8c6528", "content-length": "3141", "date": "Wed, 30 Apr 2025 06:44:16 GMT"} })
122192025-04-30T06:44:16.917ZDEBGtest_utilization (BackgroundTasks): begin collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
122202025-04-30T06:44:16.917ZDEBGtest_utilization (BackgroundTasks): end collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
122212025-04-30T06:44:16.917ZDEBGtest_utilization (BackgroundTasks): finished collection background_task = inventory_collection
122222025-04-30T06:44:16.920ZTRACtest_utilization (BackgroundTasks): authorize begin action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory
122232025-04-30T06:44:16.920ZTRACtest_utilization (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
122242025-04-30T06:44:16.920ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
122252025-04-30T06:44:16.920ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
122262025-04-30T06:44:16.923ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
122272025-04-30T06:44:16.925ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
122282025-04-30T06:44:16.933ZDEBGtest_utilization (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory result = Ok(())
122292025-04-30T06:44:16.933ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
122302025-04-30T06:44:16.933ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
122312025-04-30T06:44:16.936ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
122322025-04-30T06:44:16.936ZDEBGtest_utilization (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 47974, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }], valid_until: Instant { tv_sec: 4321, tv_nsec: 735672241 } })
122332025-04-30T06:44:16.936ZTRACtest_utilization (DnsResolver): lookup_service_targets: looking up SRV target
122342025-04-30T06:44:16.938ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
122352025-04-30T06:44:16.945ZDEBGtest_utilization (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory result = Ok(())
122362025-04-30T06:44:16.945ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
122372025-04-30T06:44:16.945ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
122382025-04-30T06:44:16.946ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
122392025-04-30T06:44:16.950ZTRACtest_utilization (dns): buffer buffer = Length: 108 (0x6c) bytes\n0000: db 4f 01 00 00 01 00 00 00 00 00 01 2d 64 65 6e .O..........-den\n0010: 64 72 69 74 65 2d 62 36 64 36 35 33 34 31 2d 31 drite-b6d65341-1\n0020: 36 37 63 2d 34 31 64 66 2d 39 62 35 63 2d 34 31 67c-41df-9b5c-41\n0030: 63 64 65 64 39 39 63 32 32 39 04 68 6f 73 74 0d cded99c229.host.\n0040: 63 6f 6e 74 72 6f 6c 2d 70 6c 61 6e 65 05 6f 78 control-plane.ox\n0050: 69 64 65 08 69 6e 74 65 72 6e 61 6c 00 00 1c 00 ide.internal....\n0060: 01 00 00 29 04 d0 00 00 00 00 00 00 ...)........ kind = dns peer_addr = [::1]:62076 req_id = c5d384bb-75d9-4fd9-a8fd-4d93f0579324
122402025-04-30T06:44:16.950ZDEBGtest_utilization (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 56143,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n query_type: AAAA,\n query_class: IN,\n },\n },\n original: [\n 45,\n 100,\n 101,\n 110,\n 100,\n 114,\n 105,\n 116,\n 101,\n 45,\n 98,\n 54,\n 100,\n 54,\n 53,\n 51,\n 52,\n 49,\n 45,\n 49,\n 54,\n 55,\n 99,\n 45,\n 52,\n 49,\n 100,\n 102,\n 45,\n 57,\n 98,\n 53,\n 99,\n 45,\n 52,\n 49,\n 99,\n 100,\n 101,\n 100,\n 57,\n 57,\n 99,\n 50,\n 50,\n 57,\n 4,\n 104,\n 111,\n 115,\n 116,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 28,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:62076 req_id = c5d384bb-75d9-4fd9-a8fd-4d93f0579324
122412025-04-30T06:44:16.950ZDEBGtest_utilization (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
122422025-04-30T06:44:16.951ZDEBGtest_utilization (dns): dns response additional_records = [] kind = dns peer_addr = [::1]:62076 query = LowerQuery { name: LowerName(Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.")), original: Query { name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), query_type: AAAA, query_class: IN } } records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] req_id = c5d384bb-75d9-4fd9-a8fd-4d93f0579324
122432025-04-30T06:44:16.951ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
122442025-04-30T06:44:16.951ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
122452025-04-30T06:44:16.954ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
122462025-04-30T06:44:16.954ZINFOtest_utilization (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager rack_id = c19a698f-c6f9-4a17-ae30-20d711b8f7dc
122472025-04-30T06:44:17.015ZINFOtest_utilization (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager rack_id = c19a698f-c6f9-4a17-ae30-20d711b8f7dc zone_address = ::1
122482025-04-30T06:44:17.016ZDEBGtest_utilization (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = c19a698f-c6f9-4a17-ae30-20d711b8f7dc uri = http://[::1]:12225/local/switch-id
122492025-04-30T06:44:17.017ZDEBGtest_utilization (MgsClient): client response background_task = switch_port_config_manager rack_id = c19a698f-c6f9-4a17-ae30-20d711b8f7dc result = Err(reqwest::Error { kind: Request, url: "http://[::1]:12225/local/switch-id", source: hyper_util::client::legacy::Error(Connect, ConnectError("tcp connect error", Os { code: 146, kind: ConnectionRefused, message: "Connection refused" })) })
122502025-04-30T06:44:17.017ZWARNtest_utilization (BackgroundTasks): failed to identify switch slot for dendrite background_task = switch_port_config_manager rack_id = c19a698f-c6f9-4a17-ae30-20d711b8f7dc reason = Communication Error: error sending request for url (http://[::1]:12225/local/switch-id): client error (Connect): tcp connect error: Connection refused (os error 146): Connection refused (os error 146) zone_address = ::1
122512025-04-30T06:44:17.020ZWARNtest_utilization (BackgroundTasks): Failed to map switch zone addr: Communication Error: error sending request for url (http://[::1]:12225/local/switch-id), retrying background_task = switch_port_config_manager rack_id = c19a698f-c6f9-4a17-ae30-20d711b8f7dc
122522025-04-30T06:44:17.021ZDEBGtest_utilization (collection-task): sent timer-based collection request to the collection task collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
122532025-04-30T06:44:17.021ZDEBGtest_utilization (collection-task): collecting from producer collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
122542025-04-30T06:44:17.024ZTRACtest_utilization (collection-task): sent collection request to producer collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
122552025-04-30T06:44:17.025ZDEBGtest_utilization (collection-task): collected results from producer collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 n_results = 1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
122562025-04-30T06:44:17.025ZTRACtest_utilization (collection-task): forwarded results to main collection loop collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
122572025-04-30T06:44:17.025ZTRACtest_utilization (collection-task): top of inner collection loop, waiting for next request collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
122582025-04-30T06:44:17.053ZWARNtest_utilization (datastore): Non-retryable transaction failure err = DatabaseError(UniqueViolation, "duplicate key value violates unique constraint \\"inv_caboose_pkey\\"") transaction = inventory_insert_collection
122592025-04-30T06:44:17.054ZWARNtest_utilization (BackgroundTasks): inventory collection failed background_task = inventory_collection error = failed to collect inventory: saving inventory to database: Internal Error: unexpected database error: duplicate key value violates unique constraint "inv_caboose_pkey"
122602025-04-30T06:44:17.054ZDEBGtest_utilization (BackgroundTasks): activation complete background_task = inventory_collection elapsed = 790.204471ms iteration = 30 status = {"error":"failed to collect inventory: saving inventory to database: Internal Error: unexpected database error: duplicate key value violates unique constraint \\"inv_caboose_pkey\\""}
122612025-04-30T06:44:17.054ZDEBGtest_utilization (BackgroundTasks): activating background_task = inventory_collection iteration = 31 reason = Signaled
122622025-04-30T06:44:17.054ZTRACtest_utilization (BackgroundTasks): authorize begin action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory
122632025-04-30T06:44:17.055ZTRACtest_utilization (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
122642025-04-30T06:44:17.055ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
122652025-04-30T06:44:17.056ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
122662025-04-30T06:44:17.057ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
122672025-04-30T06:44:17.059ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
122682025-04-30T06:44:17.066ZDEBGtest_utilization (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory result = Ok(())
122692025-04-30T06:44:17.067ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
122702025-04-30T06:44:17.067ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
122712025-04-30T06:44:17.067ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
122722025-04-30T06:44:17.073ZDEBGtest_utilization: attempt to fetch latest inventory collection result = Ok("found")
122732025-04-30T06:44:17.074ZDEBGtest_utilization (BackgroundTasks): inventory_prune_one: nothing eligible for removal (too few) background_task = inventory_collection candidates = [(13d8b8d9-6c6b-47a6-9190-952544edb75b, 0)]
122742025-04-30T06:44:17.074ZTRACtest_utilization (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal
122752025-04-30T06:44:17.075ZTRACtest_utilization (dns): buffer buffer = Length: 67 (0x43) bytes\n0000: b5 4a 01 00 00 01 00 00 00 00 00 01 04 5f 6d 67 .J..........._mg\n0010: 73 04 5f 74 63 70 0d 63 6f 6e 74 72 6f 6c 2d 70 s._tcp.control-p\n0020: 6c 61 6e 65 05 6f 78 69 64 65 08 69 6e 74 65 72 lane.oxide.inter\n0030: 6e 61 6c 00 00 21 00 01 00 00 29 04 d0 00 00 00 nal..!....).....\n0040: 00 00 00 ... kind = dns peer_addr = [::1]:54782 req_id = fa5b6ebd-9417-463c-a6be-c0007f62a15a
122762025-04-30T06:44:17.075ZDEBGtest_utilization (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 46410,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_mgs._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_mgs._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 4,\n 95,\n 109,\n 103,\n 115,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:54782 req_id = fa5b6ebd-9417-463c-a6be-c0007f62a15a
122772025-04-30T06:44:17.075ZDEBGtest_utilization (store): query key key = _mgs._tcp
122782025-04-30T06:44:17.075ZDEBGtest_utilization (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
122792025-04-30T06:44:17.075ZDEBGtest_utilization (dns): dns response additional_records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal"), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] kind = dns peer_addr = [::1]:54782 query = LowerQuery { name: LowerName(Name("_mgs._tcp.control-plane.oxide.internal.")), original: Query { name: Name("_mgs._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN } } records = [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 48759, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal") })) }] req_id = fa5b6ebd-9417-463c-a6be-c0007f62a15a
122802025-04-30T06:44:17.076ZDEBGtest_utilization (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgs._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 48759, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }], valid_until: Instant { tv_sec: 4321, tv_nsec: 874995253 } })
122812025-04-30T06:44:17.076ZTRACtest_utilization (DnsResolver): lookup_service_targets: looking up SRV target
122822025-04-30T06:44:17.076ZTRACtest_utilization (dns): buffer buffer = Length: 108 (0x6c) bytes\n0000: b6 40 01 00 00 01 00 00 00 00 00 01 2d 64 65 6e .@..........-den\n0010: 64 72 69 74 65 2d 62 36 64 36 35 33 34 31 2d 31 drite-b6d65341-1\n0020: 36 37 63 2d 34 31 64 66 2d 39 62 35 63 2d 34 31 67c-41df-9b5c-41\n0030: 63 64 65 64 39 39 63 32 32 39 04 68 6f 73 74 0d cded99c229.host.\n0040: 63 6f 6e 74 72 6f 6c 2d 70 6c 61 6e 65 05 6f 78 control-plane.ox\n0050: 69 64 65 08 69 6e 74 65 72 6e 61 6c 00 00 1c 00 ide.internal....\n0060: 01 00 00 29 04 d0 00 00 00 00 00 00 ...)........ kind = dns peer_addr = [::1]:55382 req_id = 39055b67-c45c-4840-99c6-96f753c2e887
122832025-04-30T06:44:17.076ZDEBGtest_utilization (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 46656,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n query_type: AAAA,\n query_class: IN,\n },\n },\n original: [\n 45,\n 100,\n 101,\n 110,\n 100,\n 114,\n 105,\n 116,\n 101,\n 45,\n 98,\n 54,\n 100,\n 54,\n 53,\n 51,\n 52,\n 49,\n 45,\n 49,\n 54,\n 55,\n 99,\n 45,\n 52,\n 49,\n 100,\n 102,\n 45,\n 57,\n 98,\n 53,\n 99,\n 45,\n 52,\n 49,\n 99,\n 100,\n 101,\n 100,\n 57,\n 57,\n 99,\n 50,\n 50,\n 57,\n 4,\n 104,\n 111,\n 115,\n 116,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 28,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:55382 req_id = 39055b67-c45c-4840-99c6-96f753c2e887
122842025-04-30T06:44:17.076ZDEBGtest_utilization (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
122852025-04-30T06:44:17.076ZDEBGtest_utilization (dns): dns response additional_records = [] kind = dns peer_addr = [::1]:55382 query = LowerQuery { name: LowerName(Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.")), original: Query { name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), query_type: AAAA, query_class: IN } } records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] req_id = 39055b67-c45c-4840-99c6-96f753c2e887
122862025-04-30T06:44:17.148ZTRACtest_utilization (DnsResolver): lookup_all_socket_v6 srv dns_name = _clickhouse-admin-keeper._tcp.control-plane.oxide.internal
122872025-04-30T06:44:17.149ZTRACtest_utilization (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory
122882025-04-30T06:44:17.149ZTRACtest_utilization (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
122892025-04-30T06:44:17.149ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
122902025-04-30T06:44:17.149ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
122912025-04-30T06:44:17.151ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
122922025-04-30T06:44:17.152ZTRACtest_utilization (dns): buffer buffer = Length: 87 (0x57) bytes\n0000: 37 e9 01 00 00 01 00 00 00 00 00 01 18 5f 63 6c 7............_cl\n0010: 69 63 6b 68 6f 75 73 65 2d 61 64 6d 69 6e 2d 6b ickhouse-admin-k\n0020: 65 65 70 65 72 04 5f 74 63 70 0d 63 6f 6e 74 72 eeper._tcp.contr\n0030: 6f 6c 2d 70 6c 61 6e 65 05 6f 78 69 64 65 08 69 ol-plane.oxide.i\n0040: 6e 74 65 72 6e 61 6c 00 00 21 00 01 00 00 29 04 nternal..!....).\n0050: d0 00 00 00 00 00 00 ....... kind = dns peer_addr = [::1]:52151 req_id = ed0d0958-ce0c-4e31-a393-19970f6de5e8
122932025-04-30T06:44:17.152ZDEBGtest_utilization (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 14313,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 24,\n 95,\n 99,\n 108,\n 105,\n 99,\n 107,\n 104,\n 111,\n 117,\n 115,\n 101,\n 45,\n 97,\n 100,\n 109,\n 105,\n 110,\n 45,\n 107,\n 101,\n 101,\n 112,\n 101,\n 114,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:52151 req_id = ed0d0958-ce0c-4e31-a393-19970f6de5e8
122942025-04-30T06:44:17.153ZDEBGtest_utilization (store): query key key = _clickhouse-admin-keeper._tcp
122952025-04-30T06:44:17.153ZERROtest_utilization (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 14313, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 1, }, query: WireQuery { query: LowerQuery { name: LowerName( Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), ), original: Query { name: Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN, }, }, original: [ 24, 95, 99, 108, 105, 99, 107, 104, 111, 117, 115, 101, 45, 97, 100, 109, 105, 110, 45, 107, 101, 101, 112, 101, 114, 4, 95, 116, 99, 112, 13, 99, 111, 110, 116, 114, 111, 108, 45, 112, 108, 97, 110, 101, 5, 111, 120, 105, 100, 101, 8, 105, 110, 116, 101, 114, 110, 97, 108, 0, 0, 33, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: Some( Edns { rcode_high: 0, version: 0, dnssec_ok: false, max_payload: 1232, options: OPT { options: {}, }, }, ), } NXDOMAIN: no records found for name: "_clickhouse-admin-keeper._tcp.control-plane.oxide.internal." kind = dns peer_addr = [::1]:52151 req_id = ed0d0958-ce0c-4e31-a393-19970f6de5e8
122962025-04-30T06:44:17.153ZDEBGtest_utilization (BackgroundTasks): begin collection background_task = inventory_collection
122972025-04-30T06:44:17.153ZDEBGtest_utilization (BackgroundTasks): begin collection from MGS background_task = inventory_collection mgs_url = http://[::1]:48759
122982025-04-30T06:44:17.153ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/ignition
122992025-04-30T06:44:17.155ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
123002025-04-30T06:44:17.165ZDEBGtest_utilization (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory result = Ok(())
123012025-04-30T06:44:17.166ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
123022025-04-30T06:44:17.166ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
123032025-04-30T06:44:17.166ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
123042025-04-30T06:44:17.166ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/ignition", status: 200, headers: {"content-type": "application/json", "x-request-id": "74ade131-10d4-492b-a339-5725f5b640ab", "content-length": "835", "date": "Wed, 30 Apr 2025 06:44:16 GMT"} })
123052025-04-30T06:44:17.167ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0
123062025-04-30T06:44:17.170ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
123072025-04-30T06:44:17.170ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
123082025-04-30T06:44:17.171ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
123092025-04-30T06:44:17.172ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "9931229d-e6ef-4dc4-84ec-1fd8d0914fc7", "content-length": "734", "date": "Wed, 30 Apr 2025 06:44:16 GMT"} })
123102025-04-30T06:44:17.172ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0/component/sp/caboose?firmware_slot=0
123112025-04-30T06:44:17.176ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "8b05c001-f8cc-4257-8620-d1548a4ee5c7", "content-length": "111", "date": "Wed, 30 Apr 2025 06:44:16 GMT"} })
123122025-04-30T06:44:17.176ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0/component/sp/caboose?firmware_slot=1
123132025-04-30T06:44:17.180ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "86d38bf8-0a81-4638-9903-6ae551106018", "content-length": "111", "date": "Wed, 30 Apr 2025 06:44:16 GMT"} })
123142025-04-30T06:44:17.180ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0/component/rot/caboose?firmware_slot=0
123152025-04-30T06:44:17.184ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "a8801f2b-422a-4baa-ad01-2010bf9da863", "content-length": "170", "date": "Wed, 30 Apr 2025 06:44:16 GMT"} })
123162025-04-30T06:44:17.185ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0/component/rot/caboose?firmware_slot=1
123172025-04-30T06:44:17.189ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "ccad4b83-148f-4753-a3d0-f0f15d98ed63", "content-length": "170", "date": "Wed, 30 Apr 2025 06:44:16 GMT"} })
123182025-04-30T06:44:17.189ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0/component/stage0/caboose?firmware_slot=0
123192025-04-30T06:44:17.192ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "079ed197-97c7-40eb-abad-c559f98652e3", "content-length": "179", "date": "Wed, 30 Apr 2025 06:44:16 GMT"} })
123202025-04-30T06:44:17.193ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0/component/stage0/caboose?firmware_slot=1
123212025-04-30T06:44:17.196ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "5314a0e5-9e3c-46f1-918b-ca5c3d9caee8", "content-length": "179", "date": "Wed, 30 Apr 2025 06:44:16 GMT"} })
123222025-04-30T06:44:17.196ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0/component/rot/cmpa
123232025-04-30T06:44:17.197ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "7d9c0376-16e3-40bd-9bf2-dbcc02c34c3d", "content-length": "702", "date": "Wed, 30 Apr 2025 06:44:16 GMT"} })
123242025-04-30T06:44:17.198ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0/component/rot/cfpa
123252025-04-30T06:44:17.203ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "c6e7068c-6258-41af-8ba1-d6b61c8916dc", "content-length": "727", "date": "Wed, 30 Apr 2025 06:44:16 GMT"} })
123262025-04-30T06:44:17.203ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0/component/rot/cfpa
123272025-04-30T06:44:17.205ZDEBGtest_utilization: attempt to fetch latest inventory collection result = Ok("found")
123282025-04-30T06:44:17.206ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "61e346ef-886e-41ad-97b3-734b8774ab6d", "content-length": "729", "date": "Wed, 30 Apr 2025 06:44:16 GMT"} })
123292025-04-30T06:44:17.206ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0/component/rot/cfpa
123302025-04-30T06:44:17.208ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "65053798-43ff-4443-8b44-0709796f2df2", "content-length": "728", "date": "Wed, 30 Apr 2025 06:44:16 GMT"} })
123312025-04-30T06:44:17.208ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1
123322025-04-30T06:44:17.209ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1", status: 200, headers: {"content-type": "application/json", "x-request-id": "b250ad30-148f-42d2-a675-8ae25d8afa40", "content-length": "734", "date": "Wed, 30 Apr 2025 06:44:16 GMT"} })
123332025-04-30T06:44:17.210ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1/component/sp/caboose?firmware_slot=0
123342025-04-30T06:44:17.214ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "8c895dd5-6dad-4d03-8b8b-594e492012dd", "content-length": "111", "date": "Wed, 30 Apr 2025 06:44:16 GMT"} })
123352025-04-30T06:44:17.214ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1/component/sp/caboose?firmware_slot=1
123362025-04-30T06:44:17.216ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "84b3f3c9-5221-4b77-bc69-b9f41f89c950", "content-length": "111", "date": "Wed, 30 Apr 2025 06:44:16 GMT"} })
123372025-04-30T06:44:17.216ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1/component/rot/caboose?firmware_slot=0
123382025-04-30T06:44:17.219ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "5f804615-519e-43b8-8356-e818444d01ea", "content-length": "170", "date": "Wed, 30 Apr 2025 06:44:16 GMT"} })
123392025-04-30T06:44:17.219ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1/component/rot/caboose?firmware_slot=1
123402025-04-30T06:44:17.222ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "dfaf33af-0959-4103-aa09-8d29ac8793f8", "content-length": "170", "date": "Wed, 30 Apr 2025 06:44:16 GMT"} })
123412025-04-30T06:44:17.222ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1/component/stage0/caboose?firmware_slot=0
123422025-04-30T06:44:17.225ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "915f422d-e201-44c0-9b60-bfea387dcab2", "content-length": "179", "date": "Wed, 30 Apr 2025 06:44:16 GMT"} })
123432025-04-30T06:44:17.227ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1/component/stage0/caboose?firmware_slot=1
123442025-04-30T06:44:17.230ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "0144e810-e58c-4410-93e8-085876e32611", "content-length": "179", "date": "Wed, 30 Apr 2025 06:44:16 GMT"} })
123452025-04-30T06:44:17.230ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1/component/rot/cmpa
123462025-04-30T06:44:17.231ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "a1389a6c-ef12-4e5a-8cad-429cf59bc334", "content-length": "702", "date": "Wed, 30 Apr 2025 06:44:16 GMT"} })
123472025-04-30T06:44:17.232ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1/component/rot/cfpa
123482025-04-30T06:44:17.233ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "ef6df174-f366-4950-b35a-0f0826d0acc3", "content-length": "727", "date": "Wed, 30 Apr 2025 06:44:16 GMT"} })
123492025-04-30T06:44:17.233ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1/component/rot/cfpa
123502025-04-30T06:44:17.234ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "531bd2b2-57fd-4936-9dba-5f5842a6dbde", "content-length": "729", "date": "Wed, 30 Apr 2025 06:44:16 GMT"} })
123512025-04-30T06:44:17.234ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1/component/rot/cfpa
123522025-04-30T06:44:17.235ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "0d6f4c2a-25e4-41f8-bfbb-dfb1ea0138e6", "content-length": "728", "date": "Wed, 30 Apr 2025 06:44:16 GMT"} })
123532025-04-30T06:44:17.235ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0
123542025-04-30T06:44:17.238ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "233ab03c-cce4-44f8-add6-45c0f95aeeb9", "content-length": "723", "date": "Wed, 30 Apr 2025 06:44:16 GMT"} })
123552025-04-30T06:44:17.239ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0/component/sp/caboose?firmware_slot=0
123562025-04-30T06:44:17.242ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "50d52d70-085b-4ddd-bc9d-1c92a6df1c21", "content-length": "109", "date": "Wed, 30 Apr 2025 06:44:16 GMT"} })
123572025-04-30T06:44:17.242ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0/component/sp/caboose?firmware_slot=1
123582025-04-30T06:44:17.245ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "0e0a013c-7b38-4e79-bfb6-870b49da34b2", "content-length": "109", "date": "Wed, 30 Apr 2025 06:44:16 GMT"} })
123592025-04-30T06:44:17.245ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0/component/rot/caboose?firmware_slot=0
123602025-04-30T06:44:17.248ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "95a91901-38be-44c8-aa5c-6fb30945d04e", "content-length": "169", "date": "Wed, 30 Apr 2025 06:44:16 GMT"} })
123612025-04-30T06:44:17.248ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0/component/rot/caboose?firmware_slot=1
123622025-04-30T06:44:17.250ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "6b4bba95-3104-4c71-ba3a-7601a9f2c6a6", "content-length": "169", "date": "Wed, 30 Apr 2025 06:44:16 GMT"} })
123632025-04-30T06:44:17.250ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0/component/stage0/caboose?firmware_slot=0
123642025-04-30T06:44:17.253ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "9fb83628-e642-4185-a170-ef9352159c17", "content-length": "178", "date": "Wed, 30 Apr 2025 06:44:16 GMT"} })
123652025-04-30T06:44:17.253ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0/component/stage0/caboose?firmware_slot=1
123662025-04-30T06:44:17.256ZTRACtest_utilization (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory
123672025-04-30T06:44:17.256ZTRACtest_utilization (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
123682025-04-30T06:44:17.256ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
123692025-04-30T06:44:17.256ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
123702025-04-30T06:44:17.261ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
123712025-04-30T06:44:17.264ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "5ce62e2f-4773-4b39-a34a-aacd589df2c8", "content-length": "178", "date": "Wed, 30 Apr 2025 06:44:16 GMT"} })
123722025-04-30T06:44:17.264ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0/component/rot/cmpa
123732025-04-30T06:44:17.266ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
123742025-04-30T06:44:17.276ZDEBGtest_utilization (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory result = Ok(())
123752025-04-30T06:44:17.276ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
123762025-04-30T06:44:17.276ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
123772025-04-30T06:44:17.277ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
123782025-04-30T06:44:17.277ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "2416b42f-6679-4e64-8dc4-3190cdd56b4f", "content-length": "702", "date": "Wed, 30 Apr 2025 06:44:16 GMT"} })
123792025-04-30T06:44:17.277ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0/component/rot/cfpa
123802025-04-30T06:44:17.280ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "f9906c9d-a2b3-487a-b8f5-fee3ef8c69dd", "content-length": "727", "date": "Wed, 30 Apr 2025 06:44:16 GMT"} })
123812025-04-30T06:44:17.280ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0/component/rot/cfpa
123822025-04-30T06:44:17.280ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
123832025-04-30T06:44:17.280ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
123842025-04-30T06:44:17.281ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
123852025-04-30T06:44:17.284ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "59bf6eb2-6f1b-4bce-9a02-9c29f2320b8f", "content-length": "729", "date": "Wed, 30 Apr 2025 06:44:16 GMT"} })
123862025-04-30T06:44:17.284ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0/component/rot/cfpa
123872025-04-30T06:44:17.285ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "61fbfc70-0516-4235-88cd-c8deb061ef1e", "content-length": "728", "date": "Wed, 30 Apr 2025 06:44:16 GMT"} })
123882025-04-30T06:44:17.285ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1
123892025-04-30T06:44:17.294ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1", status: 200, headers: {"content-type": "application/json", "x-request-id": "0b84eebf-8c38-4bdb-848f-66435762f4ef", "content-length": "723", "date": "Wed, 30 Apr 2025 06:44:16 GMT"} })
123902025-04-30T06:44:17.294ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1/component/sp/caboose?firmware_slot=0
123912025-04-30T06:44:17.296ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "3f02b4f8-7e45-4fe7-8b60-a4a0bac6be99", "content-length": "109", "date": "Wed, 30 Apr 2025 06:44:16 GMT"} })
123922025-04-30T06:44:17.297ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1/component/sp/caboose?firmware_slot=1
123932025-04-30T06:44:17.299ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "5414da23-b944-40a2-b84a-704d9818b578", "content-length": "109", "date": "Wed, 30 Apr 2025 06:44:16 GMT"} })
123942025-04-30T06:44:17.300ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1/component/rot/caboose?firmware_slot=0
123952025-04-30T06:44:17.306ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "4578c324-e5c9-4cbc-89bc-e1183d7dfb40", "content-length": "169", "date": "Wed, 30 Apr 2025 06:44:16 GMT"} })
123962025-04-30T06:44:17.306ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1/component/rot/caboose?firmware_slot=1
123972025-04-30T06:44:17.309ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "6ca3ff72-220a-43f8-8e76-3e73db103692", "content-length": "169", "date": "Wed, 30 Apr 2025 06:44:16 GMT"} })
123982025-04-30T06:44:17.309ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1/component/stage0/caboose?firmware_slot=0
123992025-04-30T06:44:17.311ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "9c2e82e0-7401-41ce-8240-844b8418093c", "content-length": "178", "date": "Wed, 30 Apr 2025 06:44:16 GMT"} })
124002025-04-30T06:44:17.311ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1/component/stage0/caboose?firmware_slot=1
124012025-04-30T06:44:17.315ZDEBGtest_utilization: attempt to fetch latest inventory collection result = Ok("found")
124022025-04-30T06:44:17.317ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "c75e4963-da73-4bf6-bc4b-cd2ed9e89f4a", "content-length": "178", "date": "Wed, 30 Apr 2025 06:44:16 GMT"} })
124032025-04-30T06:44:17.317ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1/component/rot/cmpa
124042025-04-30T06:44:17.318ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "55d27596-b29d-4bd0-a2ca-19c01cda9d76", "content-length": "702", "date": "Wed, 30 Apr 2025 06:44:16 GMT"} })
124052025-04-30T06:44:17.318ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1/component/rot/cfpa
124062025-04-30T06:44:17.321ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "894877fc-917b-484a-a138-0f76f4596800", "content-length": "727", "date": "Wed, 30 Apr 2025 06:44:16 GMT"} })
124072025-04-30T06:44:17.321ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1/component/rot/cfpa
124082025-04-30T06:44:17.322ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "a71ab0b1-6d73-4c4e-bb5b-ef5b31d9bfa7", "content-length": "729", "date": "Wed, 30 Apr 2025 06:44:16 GMT"} })
124092025-04-30T06:44:17.323ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1/component/rot/cfpa
124102025-04-30T06:44:17.325ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "d6ca2d10-e15d-400f-93a6-aea9aee73a68", "content-length": "728", "date": "Wed, 30 Apr 2025 06:44:16 GMT"} })
124112025-04-30T06:44:17.325ZTRACtest_utilization (BackgroundTasks): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet
124122025-04-30T06:44:17.325ZTRACtest_utilization (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
124132025-04-30T06:44:17.325ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
124142025-04-30T06:44:17.325ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
124152025-04-30T06:44:17.326ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
124162025-04-30T06:44:17.338ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
124172025-04-30T06:44:17.346ZDEBGtest_utilization (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet result = Ok(())
124182025-04-30T06:44:17.346ZTRACtest_utilization (BackgroundTasks): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet
124192025-04-30T06:44:17.346ZTRACtest_utilization (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
124202025-04-30T06:44:17.346ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
124212025-04-30T06:44:17.346ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
124222025-04-30T06:44:17.347ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
124232025-04-30T06:44:17.350ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
124242025-04-30T06:44:17.364ZDEBGtest_utilization (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet result = Ok(())
124252025-04-30T06:44:17.364ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
124262025-04-30T06:44:17.364ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
124272025-04-30T06:44:17.365ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
124282025-04-30T06:44:17.365ZDEBGtest_utilization (results-sink): inserting 1 samples into database collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
124292025-04-30T06:44:17.365ZDEBGtest_utilization (results-sink-cluster): inserting 1 samples into database collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
124302025-04-30T06:44:17.366ZDEBGtest_utilization (clickhouse-client): unrolling 1 total samples collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 4d1485f0-110e-4161-9a33-c4610d68be5d
124312025-04-30T06:44:17.367ZTRACtest_utilization (clickhouse-client): inserting data collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 4d1485f0-110e-4161-9a33-c4610d68be5d n_columns = 4 n_rows = 2 sql = INSERT INTO oximeter.fields_string FORMAT Native
124322025-04-30T06:44:17.369ZTRACtest_utilization (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory
124332025-04-30T06:44:17.369ZTRACtest_utilization (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
124342025-04-30T06:44:17.369ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
124352025-04-30T06:44:17.369ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
124362025-04-30T06:44:17.371ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
124372025-04-30T06:44:17.446ZDEBGtest_utilization (BackgroundTasks): begin collection from Sled Agent background_task = inventory_collection sled_agent_url = http://[::1]:39365
124382025-04-30T06:44:17.447ZDEBGtest_utilization (BackgroundTasks): client request SledAgent = http://[::1]:39365 background_task = inventory_collection body = None method = GET uri = http://[::1]:39365/inventory
124392025-04-30T06:44:17.447ZINFOtest_utilization (dropshot): accepted connection local_addr = [::1]:39365 remote_addr = [::1]:36292 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
124402025-04-30T06:44:17.448ZDEBGtest_utilization (clickhouse-client): inserted rows into table collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 4d1485f0-110e-4161-9a33-c4610d68be5d n_rows = 2 table_name = fields_string
124412025-04-30T06:44:17.448ZTRACtest_utilization (clickhouse-client): inserting data collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 4d1485f0-110e-4161-9a33-c4610d68be5d n_columns = 4 n_rows = 1 sql = INSERT INTO oximeter.measurements_i64 FORMAT Native
124422025-04-30T06:44:17.449ZTRACtest_utilization (dropshot): incoming request local_addr = [::1]:39365 method = GET remote_addr = [::1]:36292 req_id = c2416031-4ae3-41e8-bc44-c9228b5e7240 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /inventory
124432025-04-30T06:44:17.449ZINFOtest_utilization (dropshot): request completed latency_us = 748 local_addr = [::1]:39365 method = GET remote_addr = [::1]:36292 req_id = c2416031-4ae3-41e8-bc44-c9228b5e7240 response_code = 200 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /inventory
124442025-04-30T06:44:17.450ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
124452025-04-30T06:44:17.460ZDEBGtest_utilization (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory result = Ok(())
124462025-04-30T06:44:17.460ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
124472025-04-30T06:44:17.460ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
124482025-04-30T06:44:17.461ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
124492025-04-30T06:44:17.462ZDEBGtest_utilization (BackgroundTasks): client response SledAgent = http://[::1]:39365 background_task = inventory_collection result = Ok(Response { url: "http://[::1]:39365/inventory", status: 200, headers: {"content-type": "application/json", "x-request-id": "c2416031-4ae3-41e8-bc44-c9228b5e7240", "content-length": "3141", "date": "Wed, 30 Apr 2025 06:44:16 GMT"} })
124502025-04-30T06:44:17.465ZDEBGtest_utilization (BackgroundTasks): begin collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
124512025-04-30T06:44:17.465ZDEBGtest_utilization (BackgroundTasks): end collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
124522025-04-30T06:44:17.465ZDEBGtest_utilization (BackgroundTasks): finished collection background_task = inventory_collection
124532025-04-30T06:44:17.467ZTRACtest_utilization (BackgroundTasks): authorize begin action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory
124542025-04-30T06:44:17.467ZTRACtest_utilization (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
124552025-04-30T06:44:17.467ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
124562025-04-30T06:44:17.467ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
124572025-04-30T06:44:17.468ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
124582025-04-30T06:44:17.469ZDEBGtest_utilization (clickhouse-client): inserted rows into table collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 4d1485f0-110e-4161-9a33-c4610d68be5d n_rows = 1 table_name = measurements_i64
124592025-04-30T06:44:17.469ZTRACtest_utilization (results-sink): successfully inserted samples collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
124602025-04-30T06:44:17.470ZWARNtest_utilization (results-sink-cluster): failed to insert some results into metric DB collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 error = Failed to check out connection to database: No backends found for this service file = oximeter/collector/src/results_sink.rs:92
124612025-04-30T06:44:17.472ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
124622025-04-30T06:44:17.472ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
124632025-04-30T06:44:17.473ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
124642025-04-30T06:44:17.473ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
124652025-04-30T06:44:17.478ZDEBGtest_utilization (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory result = Ok(())
124662025-04-30T06:44:17.478ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
124672025-04-30T06:44:17.478ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
124682025-04-30T06:44:17.478ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
124692025-04-30T06:44:17.513ZWARNtest_utilization (datastore): Non-retryable transaction failure err = DatabaseError(UniqueViolation, "duplicate key value violates unique constraint \\"inv_caboose_pkey\\"") transaction = inventory_insert_collection
124702025-04-30T06:44:17.513ZWARNtest_utilization (BackgroundTasks): inventory collection failed background_task = inventory_collection error = failed to collect inventory: saving inventory to database: Internal Error: unexpected database error: duplicate key value violates unique constraint "inv_caboose_pkey"
124712025-04-30T06:44:17.513ZDEBGtest_utilization (BackgroundTasks): activation complete background_task = inventory_collection elapsed = 458.563117ms iteration = 31 status = {"error":"failed to collect inventory: saving inventory to database: Internal Error: unexpected database error: duplicate key value violates unique constraint \\"inv_caboose_pkey\\""}
124722025-04-30T06:44:17.513ZDEBGtest_utilization (BackgroundTasks): activating background_task = inventory_collection iteration = 32 reason = Signaled
124732025-04-30T06:44:17.513ZTRACtest_utilization (BackgroundTasks): authorize begin action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory
124742025-04-30T06:44:17.513ZTRACtest_utilization (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
124752025-04-30T06:44:17.513ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
124762025-04-30T06:44:17.513ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
124772025-04-30T06:44:17.514ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
124782025-04-30T06:44:17.521ZDEBGtest_utilization: attempt to fetch latest inventory collection result = Ok("found")
124792025-04-30T06:44:17.522ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
124802025-04-30T06:44:17.526ZDEBGtest_utilization (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory result = Ok(())
124812025-04-30T06:44:17.526ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
124822025-04-30T06:44:17.526ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
124832025-04-30T06:44:17.526ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
124842025-04-30T06:44:17.532ZDEBGtest_utilization (BackgroundTasks): inventory_prune_one: nothing eligible for removal (too few) background_task = inventory_collection candidates = [(13d8b8d9-6c6b-47a6-9190-952544edb75b, 0)]
124852025-04-30T06:44:17.532ZTRACtest_utilization (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal
124862025-04-30T06:44:17.533ZTRACtest_utilization (dns): buffer buffer = Length: 67 (0x43) bytes\n0000: 97 fc 01 00 00 01 00 00 00 00 00 01 04 5f 6d 67 ............._mg\n0010: 73 04 5f 74 63 70 0d 63 6f 6e 74 72 6f 6c 2d 70 s._tcp.control-p\n0020: 6c 61 6e 65 05 6f 78 69 64 65 08 69 6e 74 65 72 lane.oxide.inter\n0030: 6e 61 6c 00 00 21 00 01 00 00 29 04 d0 00 00 00 nal..!....).....\n0040: 00 00 00 ... kind = dns peer_addr = [::1]:64546 req_id = e09b9b99-4e86-4ea5-b150-cbbb39ff29c7
124872025-04-30T06:44:17.533ZDEBGtest_utilization (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 38908,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_mgs._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_mgs._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 4,\n 95,\n 109,\n 103,\n 115,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:64546 req_id = e09b9b99-4e86-4ea5-b150-cbbb39ff29c7
124882025-04-30T06:44:17.533ZDEBGtest_utilization (store): query key key = _mgs._tcp
124892025-04-30T06:44:17.533ZDEBGtest_utilization (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
124902025-04-30T06:44:17.533ZDEBGtest_utilization (dns): dns response additional_records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal"), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] kind = dns peer_addr = [::1]:64546 query = LowerQuery { name: LowerName(Name("_mgs._tcp.control-plane.oxide.internal.")), original: Query { name: Name("_mgs._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN } } records = [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 48759, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal") })) }] req_id = e09b9b99-4e86-4ea5-b150-cbbb39ff29c7
124912025-04-30T06:44:17.534ZDEBGtest_utilization (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgs._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 48759, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }], valid_until: Instant { tv_sec: 4322, tv_nsec: 333289836 } })
124922025-04-30T06:44:17.534ZTRACtest_utilization (DnsResolver): lookup_service_targets: looking up SRV target
124932025-04-30T06:44:17.534ZTRACtest_utilization (dns): buffer buffer = Length: 108 (0x6c) bytes\n0000: bf fb 01 00 00 01 00 00 00 00 00 01 2d 64 65 6e ............-den\n0010: 64 72 69 74 65 2d 62 36 64 36 35 33 34 31 2d 31 drite-b6d65341-1\n0020: 36 37 63 2d 34 31 64 66 2d 39 62 35 63 2d 34 31 67c-41df-9b5c-41\n0030: 63 64 65 64 39 39 63 32 32 39 04 68 6f 73 74 0d cded99c229.host.\n0040: 63 6f 6e 74 72 6f 6c 2d 70 6c 61 6e 65 05 6f 78 control-plane.ox\n0050: 69 64 65 08 69 6e 74 65 72 6e 61 6c 00 00 1c 00 ide.internal....\n0060: 01 00 00 29 04 d0 00 00 00 00 00 00 ...)........ kind = dns peer_addr = [::1]:51443 req_id = df3b073f-df95-47ef-a4b3-2d5789a9aae2
124942025-04-30T06:44:17.534ZDEBGtest_utilization (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 49147,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n query_type: AAAA,\n query_class: IN,\n },\n },\n original: [\n 45,\n 100,\n 101,\n 110,\n 100,\n 114,\n 105,\n 116,\n 101,\n 45,\n 98,\n 54,\n 100,\n 54,\n 53,\n 51,\n 52,\n 49,\n 45,\n 49,\n 54,\n 55,\n 99,\n 45,\n 52,\n 49,\n 100,\n 102,\n 45,\n 57,\n 98,\n 53,\n 99,\n 45,\n 52,\n 49,\n 99,\n 100,\n 101,\n 100,\n 57,\n 57,\n 99,\n 50,\n 50,\n 57,\n 4,\n 104,\n 111,\n 115,\n 116,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 28,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:51443 req_id = df3b073f-df95-47ef-a4b3-2d5789a9aae2
124952025-04-30T06:44:17.535ZDEBGtest_utilization (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
124962025-04-30T06:44:17.535ZDEBGtest_utilization (dns): dns response additional_records = [] kind = dns peer_addr = [::1]:51443 query = LowerQuery { name: LowerName(Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.")), original: Query { name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), query_type: AAAA, query_class: IN } } records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] req_id = df3b073f-df95-47ef-a4b3-2d5789a9aae2
124972025-04-30T06:44:17.582ZTRACtest_utilization (DnsResolver): lookup_all_socket_v6 srv dns_name = _clickhouse-admin-keeper._tcp.control-plane.oxide.internal
124982025-04-30T06:44:17.582ZTRACtest_utilization (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory
124992025-04-30T06:44:17.582ZTRACtest_utilization (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
125002025-04-30T06:44:17.583ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
125012025-04-30T06:44:17.583ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
125022025-04-30T06:44:17.583ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
125032025-04-30T06:44:17.584ZTRACtest_utilization (dns): buffer buffer = Length: 87 (0x57) bytes\n0000: 3c 2a 01 00 00 01 00 00 00 00 00 01 18 5f 63 6c <*..........._cl\n0010: 69 63 6b 68 6f 75 73 65 2d 61 64 6d 69 6e 2d 6b ickhouse-admin-k\n0020: 65 65 70 65 72 04 5f 74 63 70 0d 63 6f 6e 74 72 eeper._tcp.contr\n0030: 6f 6c 2d 70 6c 61 6e 65 05 6f 78 69 64 65 08 69 ol-plane.oxide.i\n0040: 6e 74 65 72 6e 61 6c 00 00 21 00 01 00 00 29 04 nternal..!....).\n0050: d0 00 00 00 00 00 00 ....... kind = dns peer_addr = [::1]:50624 req_id = 8bc72221-fb44-417c-88b5-249e339d1be8
125042025-04-30T06:44:17.584ZDEBGtest_utilization (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 15402,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 24,\n 95,\n 99,\n 108,\n 105,\n 99,\n 107,\n 104,\n 111,\n 117,\n 115,\n 101,\n 45,\n 97,\n 100,\n 109,\n 105,\n 110,\n 45,\n 107,\n 101,\n 101,\n 112,\n 101,\n 114,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:50624 req_id = 8bc72221-fb44-417c-88b5-249e339d1be8
125052025-04-30T06:44:17.584ZDEBGtest_utilization (store): query key key = _clickhouse-admin-keeper._tcp
125062025-04-30T06:44:17.584ZERROtest_utilization (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 15402, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 1, }, query: WireQuery { query: LowerQuery { name: LowerName( Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), ), original: Query { name: Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN, }, }, original: [ 24, 95, 99, 108, 105, 99, 107, 104, 111, 117, 115, 101, 45, 97, 100, 109, 105, 110, 45, 107, 101, 101, 112, 101, 114, 4, 95, 116, 99, 112, 13, 99, 111, 110, 116, 114, 111, 108, 45, 112, 108, 97, 110, 101, 5, 111, 120, 105, 100, 101, 8, 105, 110, 116, 101, 114, 110, 97, 108, 0, 0, 33, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: Some( Edns { rcode_high: 0, version: 0, dnssec_ok: false, max_payload: 1232, options: OPT { options: {}, }, }, ), } NXDOMAIN: no records found for name: "_clickhouse-admin-keeper._tcp.control-plane.oxide.internal." kind = dns peer_addr = [::1]:50624 req_id = 8bc72221-fb44-417c-88b5-249e339d1be8
125072025-04-30T06:44:17.585ZDEBGtest_utilization (BackgroundTasks): begin collection background_task = inventory_collection
125082025-04-30T06:44:17.585ZDEBGtest_utilization (BackgroundTasks): begin collection from MGS background_task = inventory_collection mgs_url = http://[::1]:48759
125092025-04-30T06:44:17.585ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/ignition
125102025-04-30T06:44:17.589ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
125112025-04-30T06:44:17.594ZDEBGtest_utilization (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory result = Ok(())
125122025-04-30T06:44:17.594ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
125132025-04-30T06:44:17.594ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
125142025-04-30T06:44:17.595ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
125152025-04-30T06:44:17.602ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
125162025-04-30T06:44:17.602ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
125172025-04-30T06:44:17.602ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
125182025-04-30T06:44:17.609ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/ignition", status: 200, headers: {"content-type": "application/json", "x-request-id": "49b5266e-bedb-4591-b1e2-e8176dd83007", "content-length": "835", "date": "Wed, 30 Apr 2025 06:44:16 GMT"} })
125192025-04-30T06:44:17.609ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0
125202025-04-30T06:44:17.629ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "ad1fd409-50a3-41cb-9e72-ad681df911b1", "content-length": "734", "date": "Wed, 30 Apr 2025 06:44:16 GMT"} })
125212025-04-30T06:44:17.631ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0/component/sp/caboose?firmware_slot=0
125222025-04-30T06:44:17.647ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "b60eb074-22d2-46fb-b821-ae3fc78b10aa", "content-length": "111", "date": "Wed, 30 Apr 2025 06:44:16 GMT"} })
125232025-04-30T06:44:17.647ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0/component/sp/caboose?firmware_slot=1
125242025-04-30T06:44:17.654ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "f49e2f91-0327-414d-8ce1-3faca52559b0", "content-length": "111", "date": "Wed, 30 Apr 2025 06:44:16 GMT"} })
125252025-04-30T06:44:17.654ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0/component/rot/caboose?firmware_slot=0
125262025-04-30T06:44:17.659ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "4ccbdf8a-def5-49b4-8a6a-8588b3be67ef", "content-length": "170", "date": "Wed, 30 Apr 2025 06:44:16 GMT"} })
125272025-04-30T06:44:17.659ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0/component/rot/caboose?firmware_slot=1
125282025-04-30T06:44:17.667ZDEBGtest_utilization: attempt to fetch latest inventory collection result = Ok("found")
125292025-04-30T06:44:17.670ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "c861695c-eda6-4043-a5ab-cf36e901f798", "content-length": "170", "date": "Wed, 30 Apr 2025 06:44:16 GMT"} })
125302025-04-30T06:44:17.671ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0/component/stage0/caboose?firmware_slot=0
125312025-04-30T06:44:17.674ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "00d9b74a-164f-4355-8e2a-5703e4379d30", "content-length": "179", "date": "Wed, 30 Apr 2025 06:44:16 GMT"} })
125322025-04-30T06:44:17.675ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0/component/stage0/caboose?firmware_slot=1
125332025-04-30T06:44:17.679ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "9b731ffb-5373-4b59-9d7d-1ce81efe6051", "content-length": "179", "date": "Wed, 30 Apr 2025 06:44:16 GMT"} })
125342025-04-30T06:44:17.679ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0/component/rot/cmpa
125352025-04-30T06:44:17.682ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "6dfaac35-4697-4138-8262-c47c8b71d4ea", "content-length": "702", "date": "Wed, 30 Apr 2025 06:44:16 GMT"} })
125362025-04-30T06:44:17.682ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0/component/rot/cfpa
125372025-04-30T06:44:17.684ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "f465bc9e-9d04-4353-ba50-171120e39733", "content-length": "727", "date": "Wed, 30 Apr 2025 06:44:16 GMT"} })
125382025-04-30T06:44:17.684ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0/component/rot/cfpa
125392025-04-30T06:44:17.686ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "d451f26f-64e3-4016-88b5-789431a0ecf0", "content-length": "729", "date": "Wed, 30 Apr 2025 06:44:16 GMT"} })
125402025-04-30T06:44:17.686ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0/component/rot/cfpa
125412025-04-30T06:44:17.690ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "19745281-32d1-49a0-833b-011e318ffde3", "content-length": "728", "date": "Wed, 30 Apr 2025 06:44:16 GMT"} })
125422025-04-30T06:44:17.690ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1
125432025-04-30T06:44:17.693ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1", status: 200, headers: {"content-type": "application/json", "x-request-id": "7f1718e1-9803-4fe2-95fe-f35b289d71b4", "content-length": "734", "date": "Wed, 30 Apr 2025 06:44:16 GMT"} })
125442025-04-30T06:44:17.693ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1/component/sp/caboose?firmware_slot=0
125452025-04-30T06:44:17.697ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "40c49abc-533f-4afa-aabf-dbdda106c22b", "content-length": "111", "date": "Wed, 30 Apr 2025 06:44:16 GMT"} })
125462025-04-30T06:44:17.697ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1/component/sp/caboose?firmware_slot=1
125472025-04-30T06:44:17.702ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "e3383a26-c995-4cfa-b44d-f219cedca9f0", "content-length": "111", "date": "Wed, 30 Apr 2025 06:44:16 GMT"} })
125482025-04-30T06:44:17.703ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1/component/rot/caboose?firmware_slot=0
125492025-04-30T06:44:17.707ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "a4834785-be29-4573-b21a-a5b90de6504a", "content-length": "170", "date": "Wed, 30 Apr 2025 06:44:16 GMT"} })
125502025-04-30T06:44:17.707ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1/component/rot/caboose?firmware_slot=1
125512025-04-30T06:44:17.708ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "f121734b-739f-4858-9de9-5c27dac294c1", "content-length": "170", "date": "Wed, 30 Apr 2025 06:44:16 GMT"} })
125522025-04-30T06:44:17.709ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1/component/stage0/caboose?firmware_slot=0
125532025-04-30T06:44:17.711ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "101d5df5-9c7d-4cc4-a7a1-c4e97548fdee", "content-length": "179", "date": "Wed, 30 Apr 2025 06:44:16 GMT"} })
125542025-04-30T06:44:17.711ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1/component/stage0/caboose?firmware_slot=1
125552025-04-30T06:44:17.713ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "665bf599-fa94-4e64-9eda-a791dd217417", "content-length": "179", "date": "Wed, 30 Apr 2025 06:44:16 GMT"} })
125562025-04-30T06:44:17.713ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1/component/rot/cmpa
125572025-04-30T06:44:17.713ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "eaaaa1b4-8b62-4301-9796-c1c12483fe68", "content-length": "702", "date": "Wed, 30 Apr 2025 06:44:16 GMT"} })
125582025-04-30T06:44:17.713ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1/component/rot/cfpa
125592025-04-30T06:44:17.714ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "7a0b28b3-1cc7-45d6-9e24-4eacc6d1c5f2", "content-length": "727", "date": "Wed, 30 Apr 2025 06:44:16 GMT"} })
125602025-04-30T06:44:17.715ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1/component/rot/cfpa
125612025-04-30T06:44:17.715ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "20af81ce-e358-4624-a319-c37ce5b23a17", "content-length": "729", "date": "Wed, 30 Apr 2025 06:44:16 GMT"} })
125622025-04-30T06:44:17.716ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1/component/rot/cfpa
125632025-04-30T06:44:17.717ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "f2ea3323-5b45-46eb-b1ea-461ea0ad6c35", "content-length": "728", "date": "Wed, 30 Apr 2025 06:44:16 GMT"} })
125642025-04-30T06:44:17.717ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0
125652025-04-30T06:44:17.718ZTRACtest_utilization (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory
125662025-04-30T06:44:17.719ZTRACtest_utilization (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
125672025-04-30T06:44:17.719ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
125682025-04-30T06:44:17.719ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
125692025-04-30T06:44:17.721ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
125702025-04-30T06:44:17.723ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "090d3290-496b-406e-864c-a73c7e2c1db7", "content-length": "723", "date": "Wed, 30 Apr 2025 06:44:16 GMT"} })
125712025-04-30T06:44:17.723ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0/component/sp/caboose?firmware_slot=0
125722025-04-30T06:44:17.724ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
125732025-04-30T06:44:17.731ZDEBGtest_utilization (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory result = Ok(())
125742025-04-30T06:44:17.731ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
125752025-04-30T06:44:17.731ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
125762025-04-30T06:44:17.732ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
125772025-04-30T06:44:17.733ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "aca72573-79a2-44fe-8211-28535f49852c", "content-length": "109", "date": "Wed, 30 Apr 2025 06:44:16 GMT"} })
125782025-04-30T06:44:17.734ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0/component/sp/caboose?firmware_slot=1
125792025-04-30T06:44:17.734ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
125802025-04-30T06:44:17.734ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
125812025-04-30T06:44:17.735ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
125822025-04-30T06:44:17.738ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "c2f224c4-f80b-4f4e-8972-a0cea0cf9fb6", "content-length": "109", "date": "Wed, 30 Apr 2025 06:44:16 GMT"} })
125832025-04-30T06:44:17.738ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0/component/rot/caboose?firmware_slot=0
125842025-04-30T06:44:17.742ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "a011d51d-9cda-4438-ba0a-ae5a1bf90355", "content-length": "169", "date": "Wed, 30 Apr 2025 06:44:16 GMT"} })
125852025-04-30T06:44:17.742ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0/component/rot/caboose?firmware_slot=1
125862025-04-30T06:44:17.745ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "c98c250d-73d1-4692-afd0-85cd9b16d8e8", "content-length": "169", "date": "Wed, 30 Apr 2025 06:44:16 GMT"} })
125872025-04-30T06:44:17.746ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0/component/stage0/caboose?firmware_slot=0
125882025-04-30T06:44:17.757ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "db49d561-755f-4f39-a009-e9f3b97a3953", "content-length": "178", "date": "Wed, 30 Apr 2025 06:44:16 GMT"} })
125892025-04-30T06:44:17.757ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0/component/stage0/caboose?firmware_slot=1
125902025-04-30T06:44:17.771ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "199da015-2544-4b25-941a-54323dd62132", "content-length": "178", "date": "Wed, 30 Apr 2025 06:44:16 GMT"} })
125912025-04-30T06:44:17.772ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0/component/rot/cmpa
125922025-04-30T06:44:17.773ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "bf9defe6-56a7-422b-8d91-e8c75832d98c", "content-length": "702", "date": "Wed, 30 Apr 2025 06:44:16 GMT"} })
125932025-04-30T06:44:17.774ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0/component/rot/cfpa
125942025-04-30T06:44:17.775ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "ba35529c-d0c6-49cd-8fbf-2c01a4e6a20b", "content-length": "727", "date": "Wed, 30 Apr 2025 06:44:16 GMT"} })
125952025-04-30T06:44:17.775ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0/component/rot/cfpa
125962025-04-30T06:44:17.776ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "9e2ba44a-17a2-464a-8983-bb648fd787bd", "content-length": "729", "date": "Wed, 30 Apr 2025 06:44:16 GMT"} })
125972025-04-30T06:44:17.776ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0/component/rot/cfpa
125982025-04-30T06:44:17.777ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "bbdad6d1-5c35-42cf-8fb7-2c00d77c8d66", "content-length": "728", "date": "Wed, 30 Apr 2025 06:44:16 GMT"} })
125992025-04-30T06:44:17.777ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1
126002025-04-30T06:44:17.779ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1", status: 200, headers: {"content-type": "application/json", "x-request-id": "e13e0aa4-6fc4-4bdb-b72a-aa44ede4d12c", "content-length": "723", "date": "Wed, 30 Apr 2025 06:44:16 GMT"} })
126012025-04-30T06:44:17.779ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1/component/sp/caboose?firmware_slot=0
126022025-04-30T06:44:17.782ZDEBGtest_utilization: attempt to fetch latest inventory collection result = Ok("found")
126032025-04-30T06:44:17.783ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "80f78219-caf4-4146-87c4-33bd61e1f45f", "content-length": "109", "date": "Wed, 30 Apr 2025 06:44:16 GMT"} })
126042025-04-30T06:44:17.783ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1/component/sp/caboose?firmware_slot=1
126052025-04-30T06:44:17.792ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "8a5e22a7-f7b0-40a8-85ba-63dc697e7881", "content-length": "109", "date": "Wed, 30 Apr 2025 06:44:16 GMT"} })
126062025-04-30T06:44:17.793ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1/component/rot/caboose?firmware_slot=0
126072025-04-30T06:44:17.795ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "3f2c87e6-8021-4a26-abe2-4895c9735a2d", "content-length": "169", "date": "Wed, 30 Apr 2025 06:44:16 GMT"} })
126082025-04-30T06:44:17.795ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1/component/rot/caboose?firmware_slot=1
126092025-04-30T06:44:17.797ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "ff8103df-ef0e-46d7-8a92-bda02a99452b", "content-length": "169", "date": "Wed, 30 Apr 2025 06:44:16 GMT"} })
126102025-04-30T06:44:17.797ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1/component/stage0/caboose?firmware_slot=0
126112025-04-30T06:44:17.800ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "c5b8daaf-9220-4e00-be21-82f10b2d4f60", "content-length": "178", "date": "Wed, 30 Apr 2025 06:44:16 GMT"} })
126122025-04-30T06:44:17.800ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1/component/stage0/caboose?firmware_slot=1
126132025-04-30T06:44:17.802ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "fd657890-a6a6-4412-93ba-0e3b4169ceef", "content-length": "178", "date": "Wed, 30 Apr 2025 06:44:16 GMT"} })
126142025-04-30T06:44:17.803ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1/component/rot/cmpa
126152025-04-30T06:44:17.803ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "9520b294-eeb6-49b9-a33f-19349a0c5822", "content-length": "702", "date": "Wed, 30 Apr 2025 06:44:16 GMT"} })
126162025-04-30T06:44:17.804ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1/component/rot/cfpa
126172025-04-30T06:44:17.805ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "8f69ee21-05fc-4f49-9721-3f9146342ded", "content-length": "727", "date": "Wed, 30 Apr 2025 06:44:16 GMT"} })
126182025-04-30T06:44:17.805ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1/component/rot/cfpa
126192025-04-30T06:44:17.808ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "6f5e39fb-4562-453a-b935-8838660f46ee", "content-length": "729", "date": "Wed, 30 Apr 2025 06:44:16 GMT"} })
126202025-04-30T06:44:17.808ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1/component/rot/cfpa
126212025-04-30T06:44:17.809ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "b1f145ed-a703-4308-acb3-fc54c2e253f8", "content-length": "728", "date": "Wed, 30 Apr 2025 06:44:16 GMT"} })
126222025-04-30T06:44:17.810ZTRACtest_utilization (BackgroundTasks): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet
126232025-04-30T06:44:17.810ZTRACtest_utilization (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
126242025-04-30T06:44:17.810ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
126252025-04-30T06:44:17.810ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
126262025-04-30T06:44:17.811ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
126272025-04-30T06:44:17.815ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
126282025-04-30T06:44:17.819ZDEBGtest_utilization (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet result = Ok(())
126292025-04-30T06:44:17.819ZTRACtest_utilization (BackgroundTasks): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet
126302025-04-30T06:44:17.819ZTRACtest_utilization (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
126312025-04-30T06:44:17.819ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
126322025-04-30T06:44:17.819ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
126332025-04-30T06:44:17.819ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
126342025-04-30T06:44:17.825ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
126352025-04-30T06:44:17.831ZDEBGtest_utilization (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet result = Ok(())
126362025-04-30T06:44:17.831ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
126372025-04-30T06:44:17.831ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
126382025-04-30T06:44:17.831ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
126392025-04-30T06:44:17.833ZTRACtest_utilization (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory
126402025-04-30T06:44:17.833ZTRACtest_utilization (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
126412025-04-30T06:44:17.834ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
126422025-04-30T06:44:17.834ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
126432025-04-30T06:44:17.835ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
126442025-04-30T06:44:17.838ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
126452025-04-30T06:44:17.846ZDEBGtest_utilization (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory result = Ok(())
126462025-04-30T06:44:17.846ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
126472025-04-30T06:44:17.846ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
126482025-04-30T06:44:17.846ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
126492025-04-30T06:44:17.907ZDEBGtest_utilization (BackgroundTasks): begin collection from Sled Agent background_task = inventory_collection sled_agent_url = http://[::1]:39365
126502025-04-30T06:44:17.907ZDEBGtest_utilization (BackgroundTasks): client request SledAgent = http://[::1]:39365 background_task = inventory_collection body = None method = GET uri = http://[::1]:39365/inventory
126512025-04-30T06:44:17.909ZINFOtest_utilization (dropshot): accepted connection local_addr = [::1]:39365 remote_addr = [::1]:48868 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
126522025-04-30T06:44:17.912ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
126532025-04-30T06:44:17.912ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
126542025-04-30T06:44:17.914ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
126552025-04-30T06:44:17.914ZTRACtest_utilization (dropshot): incoming request local_addr = [::1]:39365 method = GET remote_addr = [::1]:48868 req_id = 00d60fe2-4240-4c39-8617-c59cfcf6c31a sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /inventory
126562025-04-30T06:44:17.915ZINFOtest_utilization (dropshot): request completed latency_us = 712 local_addr = [::1]:39365 method = GET remote_addr = [::1]:48868 req_id = 00d60fe2-4240-4c39-8617-c59cfcf6c31a response_code = 200 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /inventory
126572025-04-30T06:44:17.916ZDEBGtest_utilization (BackgroundTasks): client response SledAgent = http://[::1]:39365 background_task = inventory_collection result = Ok(Response { url: "http://[::1]:39365/inventory", status: 200, headers: {"content-type": "application/json", "x-request-id": "00d60fe2-4240-4c39-8617-c59cfcf6c31a", "content-length": "3141", "date": "Wed, 30 Apr 2025 06:44:17 GMT"} })
126582025-04-30T06:44:17.920ZDEBGtest_utilization (BackgroundTasks): begin collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
126592025-04-30T06:44:17.920ZDEBGtest_utilization (BackgroundTasks): end collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
126602025-04-30T06:44:17.920ZDEBGtest_utilization (BackgroundTasks): finished collection background_task = inventory_collection
126612025-04-30T06:44:17.923ZTRACtest_utilization (BackgroundTasks): authorize begin action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory
126622025-04-30T06:44:17.923ZTRACtest_utilization (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
126632025-04-30T06:44:17.924ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
126642025-04-30T06:44:17.924ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
126652025-04-30T06:44:17.924ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
126662025-04-30T06:44:17.927ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
126672025-04-30T06:44:17.930ZDEBGtest_utilization (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory result = Ok(())
126682025-04-30T06:44:17.932ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
126692025-04-30T06:44:17.932ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
126702025-04-30T06:44:17.932ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
126712025-04-30T06:44:17.959ZINFOtest_utilization (oximeter-agent): refreshing list of producers from Nexus collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 file = oximeter/collector/src/agent.rs:562
126722025-04-30T06:44:17.959ZDEBGtest_utilization (oximeter): client request body = None method = GET uri = http://127.0.0.1:32956/metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?sort_by=id_ascending
126732025-04-30T06:44:17.960ZTRACtest_utilization (dropshot_internal): incoming request local_addr = 127.0.0.1:32956 method = GET remote_addr = 127.0.0.1:55797 req_id = a2555628-c924-4e6f-a54a-dbd5b229e925 uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?sort_by=id_ascending
126742025-04-30T06:44:17.961ZTRACtest_utilization (dropshot_internal): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = 127.0.0.1:32956 method = GET remote_addr = 127.0.0.1:55797 req_id = a2555628-c924-4e6f-a54a-dbd5b229e925 resource = Database uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?sort_by=id_ascending
126752025-04-30T06:44:17.961ZDEBGtest_utilization (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = 127.0.0.1:32956 method = GET remote_addr = 127.0.0.1:55797 req_id = a2555628-c924-4e6f-a54a-dbd5b229e925 roles = RoleSet { roles: {} } uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?sort_by=id_ascending
126762025-04-30T06:44:17.963ZDEBGtest_utilization (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = 127.0.0.1:32956 method = GET remote_addr = 127.0.0.1:55797 req_id = a2555628-c924-4e6f-a54a-dbd5b229e925 resource = Database result = Ok(()) uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?sort_by=id_ascending
126772025-04-30T06:44:17.965ZWARNtest_utilization (datastore): Non-retryable transaction failure err = DatabaseError(UniqueViolation, "duplicate key value violates unique constraint \\"inv_caboose_pkey\\"") transaction = inventory_insert_collection
126782025-04-30T06:44:17.967ZWARNtest_utilization (BackgroundTasks): inventory collection failed background_task = inventory_collection error = failed to collect inventory: saving inventory to database: Internal Error: unexpected database error: duplicate key value violates unique constraint "inv_caboose_pkey"
126792025-04-30T06:44:17.967ZDEBGtest_utilization (BackgroundTasks): activation complete background_task = inventory_collection elapsed = 451.068103ms iteration = 32 status = {"error":"failed to collect inventory: saving inventory to database: Internal Error: unexpected database error: duplicate key value violates unique constraint \\"inv_caboose_pkey\\""}
126802025-04-30T06:44:17.967ZDEBGtest_utilization (BackgroundTasks): activating background_task = inventory_collection iteration = 33 reason = Signaled
126812025-04-30T06:44:17.967ZTRACtest_utilization (BackgroundTasks): authorize begin action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory
126822025-04-30T06:44:17.967ZTRACtest_utilization (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
126832025-04-30T06:44:17.967ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
126842025-04-30T06:44:17.967ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
126852025-04-30T06:44:17.968ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
126862025-04-30T06:44:17.974ZDEBGtest_utilization: attempt to fetch latest inventory collection result = Ok("found")
126872025-04-30T06:44:17.975ZINFOtest_utilization (dropshot_internal): request completed latency_us = 15206 local_addr = 127.0.0.1:32956 method = GET remote_addr = 127.0.0.1:55797 req_id = a2555628-c924-4e6f-a54a-dbd5b229e925 response_code = 200 uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?sort_by=id_ascending
126882025-04-30T06:44:17.976ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
126892025-04-30T06:44:17.984ZDEBGtest_utilization (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory result = Ok(())
126902025-04-30T06:44:17.984ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
126912025-04-30T06:44:17.984ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
126922025-04-30T06:44:17.985ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
126932025-04-30T06:44:17.986ZDEBGtest_utilization (oximeter): client response result = Ok(Response { url: "http://127.0.0.1:32956/metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?sort_by=id_ascending", status: 200, headers: {"content-type": "application/json", "x-request-id": "a2555628-c924-4e6f-a54a-dbd5b229e925", "content-length": "405", "date": "Wed, 30 Apr 2025 06:44:17 GMT"} })
126942025-04-30T06:44:17.987ZDEBGtest_utilization (oximeter): client request body = None method = GET uri = http://127.0.0.1:32956/metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJhYzU1ZDhkOS1hMjk4LTQ1MzQtODk4Yi1iZWVlMWM5ZTRlNTIifX0%3D
126952025-04-30T06:44:17.987ZTRACtest_utilization (dropshot_internal): incoming request local_addr = 127.0.0.1:32956 method = GET remote_addr = 127.0.0.1:55797 req_id = 853cf75a-e2b3-4fb3-aff0-2b57cd6807cd uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJhYzU1ZDhkOS1hMjk4LTQ1MzQtODk4Yi1iZWVlMWM5ZTRlNTIifX0%3D
126962025-04-30T06:44:17.989ZTRACtest_utilization (dropshot_internal): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = 127.0.0.1:32956 method = GET remote_addr = 127.0.0.1:55797 req_id = 853cf75a-e2b3-4fb3-aff0-2b57cd6807cd resource = Database uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJhYzU1ZDhkOS1hMjk4LTQ1MzQtODk4Yi1iZWVlMWM5ZTRlNTIifX0%3D
126972025-04-30T06:44:17.990ZDEBGtest_utilization (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = 127.0.0.1:32956 method = GET remote_addr = 127.0.0.1:55797 req_id = 853cf75a-e2b3-4fb3-aff0-2b57cd6807cd roles = RoleSet { roles: {} } uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJhYzU1ZDhkOS1hMjk4LTQ1MzQtODk4Yi1iZWVlMWM5ZTRlNTIifX0%3D
126982025-04-30T06:44:17.990ZDEBGtest_utilization (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = 127.0.0.1:32956 method = GET remote_addr = 127.0.0.1:55797 req_id = 853cf75a-e2b3-4fb3-aff0-2b57cd6807cd resource = Database result = Ok(()) uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJhYzU1ZDhkOS1hMjk4LTQ1MzQtODk4Yi1iZWVlMWM5ZTRlNTIifX0%3D
126992025-04-30T06:44:17.990ZDEBGtest_utilization (BackgroundTasks): inventory_prune_one: nothing eligible for removal (too few) background_task = inventory_collection candidates = [(13d8b8d9-6c6b-47a6-9190-952544edb75b, 0)]
127002025-04-30T06:44:17.990ZTRACtest_utilization (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal
127012025-04-30T06:44:17.991ZTRACtest_utilization (dns): buffer buffer = Length: 67 (0x43) bytes\n0000: d2 78 01 00 00 01 00 00 00 00 00 01 04 5f 6d 67 .x..........._mg\n0010: 73 04 5f 74 63 70 0d 63 6f 6e 74 72 6f 6c 2d 70 s._tcp.control-p\n0020: 6c 61 6e 65 05 6f 78 69 64 65 08 69 6e 74 65 72 lane.oxide.inter\n0030: 6e 61 6c 00 00 21 00 01 00 00 29 04 d0 00 00 00 nal..!....).....\n0040: 00 00 00 ... kind = dns peer_addr = [::1]:60992 req_id = 2d1d2285-0bca-4a8b-9cbc-0de94c857314
127022025-04-30T06:44:17.991ZDEBGtest_utilization (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 53880,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_mgs._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_mgs._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 4,\n 95,\n 109,\n 103,\n 115,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:60992 req_id = 2d1d2285-0bca-4a8b-9cbc-0de94c857314
127032025-04-30T06:44:17.992ZDEBGtest_utilization (store): query key key = _mgs._tcp
127042025-04-30T06:44:17.992ZDEBGtest_utilization (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
127052025-04-30T06:44:17.992ZDEBGtest_utilization (dns): dns response additional_records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal"), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] kind = dns peer_addr = [::1]:60992 query = LowerQuery { name: LowerName(Name("_mgs._tcp.control-plane.oxide.internal.")), original: Query { name: Name("_mgs._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN } } records = [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 48759, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal") })) }] req_id = 2d1d2285-0bca-4a8b-9cbc-0de94c857314
127062025-04-30T06:44:17.993ZDEBGtest_utilization (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgs._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 48759, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }], valid_until: Instant { tv_sec: 4322, tv_nsec: 792644549 } })
127072025-04-30T06:44:17.993ZTRACtest_utilization (DnsResolver): lookup_service_targets: looking up SRV target
127082025-04-30T06:44:17.994ZINFOtest_utilization (dropshot_internal): request completed latency_us = 6813 local_addr = 127.0.0.1:32956 method = GET remote_addr = 127.0.0.1:55797 req_id = 853cf75a-e2b3-4fb3-aff0-2b57cd6807cd response_code = 200 uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJhYzU1ZDhkOS1hMjk4LTQ1MzQtODk4Yi1iZWVlMWM5ZTRlNTIifX0%3D
127092025-04-30T06:44:17.994ZDEBGtest_utilization (oximeter): client response result = Ok(Response { url: "http://127.0.0.1:32956/metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJhYzU1ZDhkOS1hMjk4LTQ1MzQtODk4Yi1iZWVlMWM5ZTRlNTIifX0%3D", status: 200, headers: {"content-type": "application/json", "x-request-id": "853cf75a-e2b3-4fb3-aff0-2b57cd6807cd", "content-length": "29", "date": "Wed, 30 Apr 2025 06:44:17 GMT"} })
127102025-04-30T06:44:17.994ZTRACtest_utilization (oximeter-agent): ignoring request to update existing metric producer, since the endpoint information is the same as the existing collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
127112025-04-30T06:44:17.994ZTRACtest_utilization (oximeter-agent): ignoring request to update existing metric producer, since the endpoint information is the same as the existing collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = ac55d8d9-a298-4534-898b-beee1c9e4e52
127122025-04-30T06:44:17.994ZINFOtest_utilization (oximeter-agent): refreshed list of producers from Nexus collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 file = oximeter/collector/src/agent.rs:637 n_current_tasks = 2 n_pruned_tasks = 0
127132025-04-30T06:44:17.995ZTRACtest_utilization (dns): buffer buffer = Length: 108 (0x6c) bytes\n0000: 87 39 01 00 00 01 00 00 00 00 00 01 2d 64 65 6e .9..........-den\n0010: 64 72 69 74 65 2d 62 36 64 36 35 33 34 31 2d 31 drite-b6d65341-1\n0020: 36 37 63 2d 34 31 64 66 2d 39 62 35 63 2d 34 31 67c-41df-9b5c-41\n0030: 63 64 65 64 39 39 63 32 32 39 04 68 6f 73 74 0d cded99c229.host.\n0040: 63 6f 6e 74 72 6f 6c 2d 70 6c 61 6e 65 05 6f 78 control-plane.ox\n0050: 69 64 65 08 69 6e 74 65 72 6e 61 6c 00 00 1c 00 ide.internal....\n0060: 01 00 00 29 04 d0 00 00 00 00 00 00 ...)........ kind = dns peer_addr = [::1]:53028 req_id = 87747d18-72e6-4526-9c96-914acc42b82f
127142025-04-30T06:44:17.995ZDEBGtest_utilization (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 34617,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n query_type: AAAA,\n query_class: IN,\n },\n },\n original: [\n 45,\n 100,\n 101,\n 110,\n 100,\n 114,\n 105,\n 116,\n 101,\n 45,\n 98,\n 54,\n 100,\n 54,\n 53,\n 51,\n 52,\n 49,\n 45,\n 49,\n 54,\n 55,\n 99,\n 45,\n 52,\n 49,\n 100,\n 102,\n 45,\n 57,\n 98,\n 53,\n 99,\n 45,\n 52,\n 49,\n 99,\n 100,\n 101,\n 100,\n 57,\n 57,\n 99,\n 50,\n 50,\n 57,\n 4,\n 104,\n 111,\n 115,\n 116,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 28,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:53028 req_id = 87747d18-72e6-4526-9c96-914acc42b82f
127152025-04-30T06:44:17.995ZDEBGtest_utilization (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
127162025-04-30T06:44:17.995ZDEBGtest_utilization (dns): dns response additional_records = [] kind = dns peer_addr = [::1]:53028 query = LowerQuery { name: LowerName(Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.")), original: Query { name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), query_type: AAAA, query_class: IN } } records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] req_id = 87747d18-72e6-4526-9c96-914acc42b82f
127172025-04-30T06:44:18.059ZTRACtest_utilization (DnsResolver): lookup_all_socket_v6 srv dns_name = _clickhouse-admin-keeper._tcp.control-plane.oxide.internal
127182025-04-30T06:44:18.060ZDEBGtest_utilization (collection-task): sent timer-based collection request to the collection task collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
127192025-04-30T06:44:18.060ZDEBGtest_utilization (collection-task): collecting from producer collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
127202025-04-30T06:44:18.060ZTRACtest_utilization (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory
127212025-04-30T06:44:18.060ZTRACtest_utilization (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
127222025-04-30T06:44:18.060ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
127232025-04-30T06:44:18.060ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
127242025-04-30T06:44:18.061ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
127252025-04-30T06:44:18.063ZTRACtest_utilization (dns): buffer buffer = Length: 87 (0x57) bytes\n0000: a3 93 01 00 00 01 00 00 00 00 00 01 18 5f 63 6c ............._cl\n0010: 69 63 6b 68 6f 75 73 65 2d 61 64 6d 69 6e 2d 6b ickhouse-admin-k\n0020: 65 65 70 65 72 04 5f 74 63 70 0d 63 6f 6e 74 72 eeper._tcp.contr\n0030: 6f 6c 2d 70 6c 61 6e 65 05 6f 78 69 64 65 08 69 ol-plane.oxide.i\n0040: 6e 74 65 72 6e 61 6c 00 00 21 00 01 00 00 29 04 nternal..!....).\n0050: d0 00 00 00 00 00 00 ....... kind = dns peer_addr = [::1]:58018 req_id = 747054a5-24d1-4877-92e7-71023d49b8cc
127262025-04-30T06:44:18.063ZDEBGtest_utilization (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 41875,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 24,\n 95,\n 99,\n 108,\n 105,\n 99,\n 107,\n 104,\n 111,\n 117,\n 115,\n 101,\n 45,\n 97,\n 100,\n 109,\n 105,\n 110,\n 45,\n 107,\n 101,\n 101,\n 112,\n 101,\n 114,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:58018 req_id = 747054a5-24d1-4877-92e7-71023d49b8cc
127272025-04-30T06:44:18.063ZDEBGtest_utilization (store): query key key = _clickhouse-admin-keeper._tcp
127282025-04-30T06:44:18.063ZERROtest_utilization (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 41875, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 1, }, query: WireQuery { query: LowerQuery { name: LowerName( Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), ), original: Query { name: Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN, }, }, original: [ 24, 95, 99, 108, 105, 99, 107, 104, 111, 117, 115, 101, 45, 97, 100, 109, 105, 110, 45, 107, 101, 101, 112, 101, 114, 4, 95, 116, 99, 112, 13, 99, 111, 110, 116, 114, 111, 108, 45, 112, 108, 97, 110, 101, 5, 111, 120, 105, 100, 101, 8, 105, 110, 116, 101, 114, 110, 97, 108, 0, 0, 33, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: Some( Edns { rcode_high: 0, version: 0, dnssec_ok: false, max_payload: 1232, options: OPT { options: {}, }, }, ), } NXDOMAIN: no records found for name: "_clickhouse-admin-keeper._tcp.control-plane.oxide.internal." kind = dns peer_addr = [::1]:58018 req_id = 747054a5-24d1-4877-92e7-71023d49b8cc
127292025-04-30T06:44:18.063ZTRACtest_utilization (collection-task): sent collection request to producer collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
127302025-04-30T06:44:18.064ZDEBGtest_utilization (collection-task): collected results from producer collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 n_results = 1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
127312025-04-30T06:44:18.064ZTRACtest_utilization (collection-task): forwarded results to main collection loop collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
127322025-04-30T06:44:18.064ZTRACtest_utilization (collection-task): top of inner collection loop, waiting for next request collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
127332025-04-30T06:44:18.066ZDEBGtest_utilization (BackgroundTasks): begin collection background_task = inventory_collection
127342025-04-30T06:44:18.066ZDEBGtest_utilization (BackgroundTasks): begin collection from MGS background_task = inventory_collection mgs_url = http://[::1]:48759
127352025-04-30T06:44:18.066ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/ignition
127362025-04-30T06:44:18.066ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
127372025-04-30T06:44:18.072ZDEBGtest_utilization (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory result = Ok(())
127382025-04-30T06:44:18.072ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
127392025-04-30T06:44:18.073ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
127402025-04-30T06:44:18.073ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
127412025-04-30T06:44:18.077ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
127422025-04-30T06:44:18.077ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
127432025-04-30T06:44:18.078ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
127442025-04-30T06:44:18.078ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/ignition", status: 200, headers: {"content-type": "application/json", "x-request-id": "67cec671-4e66-487a-93ce-25004d2af10b", "content-length": "835", "date": "Wed, 30 Apr 2025 06:44:17 GMT"} })
127452025-04-30T06:44:18.079ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0
127462025-04-30T06:44:18.082ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "ae013606-a8dc-417b-9f2c-4fb32475117c", "content-length": "734", "date": "Wed, 30 Apr 2025 06:44:17 GMT"} })
127472025-04-30T06:44:18.082ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0/component/sp/caboose?firmware_slot=0
127482025-04-30T06:44:18.087ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "6e56eb5e-4721-443f-84d0-7ac53bb473a4", "content-length": "111", "date": "Wed, 30 Apr 2025 06:44:17 GMT"} })
127492025-04-30T06:44:18.087ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0/component/sp/caboose?firmware_slot=1
127502025-04-30T06:44:18.091ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "1aa0f7f1-e373-42cb-8598-5b3e4f4815a6", "content-length": "111", "date": "Wed, 30 Apr 2025 06:44:17 GMT"} })
127512025-04-30T06:44:18.092ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0/component/rot/caboose?firmware_slot=0
127522025-04-30T06:44:18.097ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "4ad86377-d488-4b7b-93d8-43f6d8e4dddb", "content-length": "170", "date": "Wed, 30 Apr 2025 06:44:17 GMT"} })
127532025-04-30T06:44:18.097ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0/component/rot/caboose?firmware_slot=1
127542025-04-30T06:44:18.101ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "51b98d1b-1ffd-4f6e-b51b-c66ca48253d3", "content-length": "170", "date": "Wed, 30 Apr 2025 06:44:17 GMT"} })
127552025-04-30T06:44:18.101ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0/component/stage0/caboose?firmware_slot=0
127562025-04-30T06:44:18.106ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "bb3bf38f-978f-43ee-9247-03516bfafedf", "content-length": "179", "date": "Wed, 30 Apr 2025 06:44:17 GMT"} })
127572025-04-30T06:44:18.106ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0/component/stage0/caboose?firmware_slot=1
127582025-04-30T06:44:18.110ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "5988452e-175a-4347-ae0d-aa9754f34893", "content-length": "179", "date": "Wed, 30 Apr 2025 06:44:17 GMT"} })
127592025-04-30T06:44:18.110ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0/component/rot/cmpa
127602025-04-30T06:44:18.113ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "42510383-ab44-4319-89aa-21bdbc36bbfc", "content-length": "702", "date": "Wed, 30 Apr 2025 06:44:17 GMT"} })
127612025-04-30T06:44:18.113ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0/component/rot/cfpa
127622025-04-30T06:44:18.115ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "7f0e17fd-cf9a-4ad7-9c9e-bf3ff0c6c626", "content-length": "727", "date": "Wed, 30 Apr 2025 06:44:17 GMT"} })
127632025-04-30T06:44:18.115ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0/component/rot/cfpa
127642025-04-30T06:44:18.118ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "63eae9f5-8a46-48a5-a8f2-cbe90098f157", "content-length": "729", "date": "Wed, 30 Apr 2025 06:44:17 GMT"} })
127652025-04-30T06:44:18.119ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0/component/rot/cfpa
127662025-04-30T06:44:18.122ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "4dea460b-eedb-4e70-bd07-3790ef8fa932", "content-length": "728", "date": "Wed, 30 Apr 2025 06:44:17 GMT"} })
127672025-04-30T06:44:18.123ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1
127682025-04-30T06:44:18.123ZDEBGtest_utilization: attempt to fetch latest inventory collection result = Ok("found")
127692025-04-30T06:44:18.126ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1", status: 200, headers: {"content-type": "application/json", "x-request-id": "92021bac-4c44-4a1e-9aa0-f33d3a5d1045", "content-length": "734", "date": "Wed, 30 Apr 2025 06:44:17 GMT"} })
127702025-04-30T06:44:18.127ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1/component/sp/caboose?firmware_slot=0
127712025-04-30T06:44:18.130ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "bd4e8f54-91e2-4dba-8950-2fa6f2d766cc", "content-length": "111", "date": "Wed, 30 Apr 2025 06:44:17 GMT"} })
127722025-04-30T06:44:18.130ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1/component/sp/caboose?firmware_slot=1
127732025-04-30T06:44:18.134ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "5a66833a-b3aa-496c-841a-8aa0d301cdf1", "content-length": "111", "date": "Wed, 30 Apr 2025 06:44:17 GMT"} })
127742025-04-30T06:44:18.135ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1/component/rot/caboose?firmware_slot=0
127752025-04-30T06:44:18.139ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "df9d18b9-0887-4c07-bc80-331e64165eca", "content-length": "170", "date": "Wed, 30 Apr 2025 06:44:17 GMT"} })
127762025-04-30T06:44:18.140ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1/component/rot/caboose?firmware_slot=1
127772025-04-30T06:44:18.144ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "dace82d0-9b32-4c29-9233-dee216cdcb0d", "content-length": "170", "date": "Wed, 30 Apr 2025 06:44:17 GMT"} })
127782025-04-30T06:44:18.144ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1/component/stage0/caboose?firmware_slot=0
127792025-04-30T06:44:18.149ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "dca59890-dd28-4da0-9d4e-5a1554883f05", "content-length": "179", "date": "Wed, 30 Apr 2025 06:44:17 GMT"} })
127802025-04-30T06:44:18.149ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1/component/stage0/caboose?firmware_slot=1
127812025-04-30T06:44:18.154ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "cae79809-d736-4912-a11c-25905e565002", "content-length": "179", "date": "Wed, 30 Apr 2025 06:44:17 GMT"} })
127822025-04-30T06:44:18.154ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1/component/rot/cmpa
127832025-04-30T06:44:18.155ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "3877afb0-fcd7-4d85-9ff9-f02ed89dc7a5", "content-length": "702", "date": "Wed, 30 Apr 2025 06:44:17 GMT"} })
127842025-04-30T06:44:18.156ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1/component/rot/cfpa
127852025-04-30T06:44:18.161ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "f4c00d0a-ba2e-41fa-9fb9-6a7e563b5195", "content-length": "727", "date": "Wed, 30 Apr 2025 06:44:17 GMT"} })
127862025-04-30T06:44:18.163ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1/component/rot/cfpa
127872025-04-30T06:44:18.164ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "47911b7a-5c69-460d-a174-5e2deee5bc81", "content-length": "729", "date": "Wed, 30 Apr 2025 06:44:17 GMT"} })
127882025-04-30T06:44:18.164ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1/component/rot/cfpa
127892025-04-30T06:44:18.165ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "148b8f6b-d1eb-4732-a6c3-e72de4526ebb", "content-length": "728", "date": "Wed, 30 Apr 2025 06:44:17 GMT"} })
127902025-04-30T06:44:18.165ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0
127912025-04-30T06:44:18.166ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "2578bf67-b8f4-4ac2-8170-eca89007d288", "content-length": "723", "date": "Wed, 30 Apr 2025 06:44:17 GMT"} })
127922025-04-30T06:44:18.166ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0/component/sp/caboose?firmware_slot=0
127932025-04-30T06:44:18.173ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "6f5f7ee9-f5a5-460e-90e2-dfbcc38b3c57", "content-length": "109", "date": "Wed, 30 Apr 2025 06:44:17 GMT"} })
127942025-04-30T06:44:18.173ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0/component/sp/caboose?firmware_slot=1
127952025-04-30T06:44:18.174ZTRACtest_utilization (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory
127962025-04-30T06:44:18.174ZTRACtest_utilization (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
127972025-04-30T06:44:18.174ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
127982025-04-30T06:44:18.174ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
127992025-04-30T06:44:18.175ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
128002025-04-30T06:44:18.178ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "c56088b0-ef81-49ff-9b72-83fdeed0d147", "content-length": "109", "date": "Wed, 30 Apr 2025 06:44:17 GMT"} })
128012025-04-30T06:44:18.178ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0/component/rot/caboose?firmware_slot=0
128022025-04-30T06:44:18.182ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
128032025-04-30T06:44:18.187ZDEBGtest_utilization (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory result = Ok(())
128042025-04-30T06:44:18.187ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
128052025-04-30T06:44:18.187ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
128062025-04-30T06:44:18.187ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
128072025-04-30T06:44:18.188ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "e80b7fea-df9d-4af8-822b-1716d35ade4d", "content-length": "169", "date": "Wed, 30 Apr 2025 06:44:17 GMT"} })
128082025-04-30T06:44:18.196ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0/component/rot/caboose?firmware_slot=1
128092025-04-30T06:44:18.200ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
128102025-04-30T06:44:18.200ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
128112025-04-30T06:44:18.201ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
128122025-04-30T06:44:18.203ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "9814cba0-ed69-4306-85d2-2dfb557af7bb", "content-length": "169", "date": "Wed, 30 Apr 2025 06:44:17 GMT"} })
128132025-04-30T06:44:18.203ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0/component/stage0/caboose?firmware_slot=0
128142025-04-30T06:44:18.212ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "943952c5-888c-4ed0-a796-665c64dbdd55", "content-length": "178", "date": "Wed, 30 Apr 2025 06:44:17 GMT"} })
128152025-04-30T06:44:18.213ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0/component/stage0/caboose?firmware_slot=1
128162025-04-30T06:44:18.217ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "fb7332c8-9c97-4a0a-bd3e-bf1d21ec5bfb", "content-length": "178", "date": "Wed, 30 Apr 2025 06:44:17 GMT"} })
128172025-04-30T06:44:18.217ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0/component/rot/cmpa
128182025-04-30T06:44:18.219ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "d764ac5c-4804-46ca-bf42-9ac10a908800", "content-length": "702", "date": "Wed, 30 Apr 2025 06:44:17 GMT"} })
128192025-04-30T06:44:18.219ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0/component/rot/cfpa
128202025-04-30T06:44:18.223ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "f93f6af6-49a7-44f3-bbff-00f3849be011", "content-length": "727", "date": "Wed, 30 Apr 2025 06:44:17 GMT"} })
128212025-04-30T06:44:18.223ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0/component/rot/cfpa
128222025-04-30T06:44:18.224ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "401400b6-e84f-40d2-acfc-2cdc6e51347b", "content-length": "729", "date": "Wed, 30 Apr 2025 06:44:17 GMT"} })
128232025-04-30T06:44:18.225ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0/component/rot/cfpa
128242025-04-30T06:44:18.226ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "560b7ec4-eea8-4899-97eb-851a191f2110", "content-length": "728", "date": "Wed, 30 Apr 2025 06:44:17 GMT"} })
128252025-04-30T06:44:18.226ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1
128262025-04-30T06:44:18.228ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1", status: 200, headers: {"content-type": "application/json", "x-request-id": "8fd64f1c-c8b0-4a51-99b0-4ff692d4ffe4", "content-length": "723", "date": "Wed, 30 Apr 2025 06:44:17 GMT"} })
128272025-04-30T06:44:18.228ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1/component/sp/caboose?firmware_slot=0
128282025-04-30T06:44:18.231ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "4c9242e3-a30a-41f8-913e-e81f05a9f641", "content-length": "109", "date": "Wed, 30 Apr 2025 06:44:17 GMT"} })
128292025-04-30T06:44:18.232ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1/component/sp/caboose?firmware_slot=1
128302025-04-30T06:44:18.235ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "53b9ea6f-672d-408c-bb37-ccd17302e63e", "content-length": "109", "date": "Wed, 30 Apr 2025 06:44:17 GMT"} })
128312025-04-30T06:44:18.235ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1/component/rot/caboose?firmware_slot=0
128322025-04-30T06:44:18.239ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "bb18b115-1acf-4217-af99-6550690cc5a9", "content-length": "169", "date": "Wed, 30 Apr 2025 06:44:17 GMT"} })
128332025-04-30T06:44:18.239ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1/component/rot/caboose?firmware_slot=1
128342025-04-30T06:44:18.239ZDEBGtest_utilization: attempt to fetch latest inventory collection result = Ok("found")
128352025-04-30T06:44:18.242ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "86e79821-e9af-45ad-8dde-ac64ec05fe48", "content-length": "169", "date": "Wed, 30 Apr 2025 06:44:17 GMT"} })
128362025-04-30T06:44:18.242ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1/component/stage0/caboose?firmware_slot=0
128372025-04-30T06:44:18.244ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "3f164792-1fb6-4536-8ec1-94b9113e1e9c", "content-length": "178", "date": "Wed, 30 Apr 2025 06:44:17 GMT"} })
128382025-04-30T06:44:18.244ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1/component/stage0/caboose?firmware_slot=1
128392025-04-30T06:44:18.246ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "e861ded2-8000-4c1d-b5f0-251f2bec08c3", "content-length": "178", "date": "Wed, 30 Apr 2025 06:44:17 GMT"} })
128402025-04-30T06:44:18.246ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1/component/rot/cmpa
128412025-04-30T06:44:18.247ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "18e2b116-a4b9-425f-9374-ad26eeab5648", "content-length": "702", "date": "Wed, 30 Apr 2025 06:44:17 GMT"} })
128422025-04-30T06:44:18.247ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1/component/rot/cfpa
128432025-04-30T06:44:18.249ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "45df5dc1-0093-4deb-b8b8-f1488b6075d2", "content-length": "727", "date": "Wed, 30 Apr 2025 06:44:17 GMT"} })
128442025-04-30T06:44:18.249ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1/component/rot/cfpa
128452025-04-30T06:44:18.251ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "713d1759-9042-49b2-81e0-77e4e5338cd4", "content-length": "729", "date": "Wed, 30 Apr 2025 06:44:17 GMT"} })
128462025-04-30T06:44:18.251ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1/component/rot/cfpa
128472025-04-30T06:44:18.252ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "5cb3afff-bf2a-4d1a-8f33-bdf7a6cba828", "content-length": "728", "date": "Wed, 30 Apr 2025 06:44:17 GMT"} })
128482025-04-30T06:44:18.252ZTRACtest_utilization (BackgroundTasks): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet
128492025-04-30T06:44:18.252ZTRACtest_utilization (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
128502025-04-30T06:44:18.252ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
128512025-04-30T06:44:18.252ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
128522025-04-30T06:44:18.253ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
128532025-04-30T06:44:18.255ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
128542025-04-30T06:44:18.261ZDEBGtest_utilization (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet result = Ok(())
128552025-04-30T06:44:18.261ZTRACtest_utilization (BackgroundTasks): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet
128562025-04-30T06:44:18.261ZTRACtest_utilization (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
128572025-04-30T06:44:18.261ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
128582025-04-30T06:44:18.261ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
128592025-04-30T06:44:18.263ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
128602025-04-30T06:44:18.267ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
128612025-04-30T06:44:18.272ZDEBGtest_utilization (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet result = Ok(())
128622025-04-30T06:44:18.272ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
128632025-04-30T06:44:18.273ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
128642025-04-30T06:44:18.273ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
128652025-04-30T06:44:18.327ZDEBGtest_utilization (BackgroundTasks): begin collection from Sled Agent background_task = inventory_collection sled_agent_url = http://[::1]:39365
128662025-04-30T06:44:18.327ZDEBGtest_utilization (BackgroundTasks): client request SledAgent = http://[::1]:39365 background_task = inventory_collection body = None method = GET uri = http://[::1]:39365/inventory
128672025-04-30T06:44:18.328ZTRACtest_utilization (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory
128682025-04-30T06:44:18.328ZTRACtest_utilization (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
128692025-04-30T06:44:18.328ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
128702025-04-30T06:44:18.328ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
128712025-04-30T06:44:18.329ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
128722025-04-30T06:44:18.329ZINFOtest_utilization (dropshot): accepted connection local_addr = [::1]:39365 remote_addr = [::1]:61482 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
128732025-04-30T06:44:18.330ZTRACtest_utilization (dropshot): incoming request local_addr = [::1]:39365 method = GET remote_addr = [::1]:61482 req_id = f993eb7b-ec86-42be-b757-e6000586c74c sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /inventory
128742025-04-30T06:44:18.330ZINFOtest_utilization (dropshot): request completed latency_us = 620 local_addr = [::1]:39365 method = GET remote_addr = [::1]:61482 req_id = f993eb7b-ec86-42be-b757-e6000586c74c response_code = 200 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /inventory
128752025-04-30T06:44:18.332ZDEBGtest_utilization (BackgroundTasks): client response SledAgent = http://[::1]:39365 background_task = inventory_collection result = Ok(Response { url: "http://[::1]:39365/inventory", status: 200, headers: {"content-type": "application/json", "x-request-id": "f993eb7b-ec86-42be-b757-e6000586c74c", "content-length": "3141", "date": "Wed, 30 Apr 2025 06:44:17 GMT"} })
128762025-04-30T06:44:18.334ZDEBGtest_utilization (BackgroundTasks): begin collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
128772025-04-30T06:44:18.334ZDEBGtest_utilization (BackgroundTasks): end collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
128782025-04-30T06:44:18.334ZDEBGtest_utilization (BackgroundTasks): finished collection background_task = inventory_collection
128792025-04-30T06:44:18.336ZTRACtest_utilization (BackgroundTasks): authorize begin action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory
128802025-04-30T06:44:18.336ZTRACtest_utilization (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
128812025-04-30T06:44:18.336ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
128822025-04-30T06:44:18.337ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
128832025-04-30T06:44:18.338ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
128842025-04-30T06:44:18.338ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
128852025-04-30T06:44:18.344ZDEBGtest_utilization (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory result = Ok(())
128862025-04-30T06:44:18.344ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
128872025-04-30T06:44:18.344ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
128882025-04-30T06:44:18.344ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
128892025-04-30T06:44:18.347ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
128902025-04-30T06:44:18.355ZDEBGtest_utilization (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory result = Ok(())
128912025-04-30T06:44:18.355ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
128922025-04-30T06:44:18.355ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
128932025-04-30T06:44:18.356ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
128942025-04-30T06:44:18.356ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
128952025-04-30T06:44:18.356ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
128962025-04-30T06:44:18.357ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
128972025-04-30T06:44:18.357ZDEBGtest_utilization (results-sink): inserting 1 samples into database collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
128982025-04-30T06:44:18.357ZDEBGtest_utilization (results-sink-cluster): inserting 1 samples into database collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
128992025-04-30T06:44:18.358ZDEBGtest_utilization (clickhouse-client): unrolling 1 total samples collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 4d1485f0-110e-4161-9a33-c4610d68be5d
129002025-04-30T06:44:18.359ZTRACtest_utilization (clickhouse-client): inserting data collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 4d1485f0-110e-4161-9a33-c4610d68be5d n_columns = 4 n_rows = 2 sql = INSERT INTO oximeter.fields_string FORMAT Native
129012025-04-30T06:44:18.375ZDEBGtest_utilization (clickhouse-client): inserted rows into table collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 4d1485f0-110e-4161-9a33-c4610d68be5d n_rows = 2 table_name = fields_string
129022025-04-30T06:44:18.376ZTRACtest_utilization (clickhouse-client): inserting data collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 4d1485f0-110e-4161-9a33-c4610d68be5d n_columns = 4 n_rows = 1 sql = INSERT INTO oximeter.measurements_i64 FORMAT Native
129032025-04-30T06:44:18.382ZDEBGtest_utilization (clickhouse-client): inserted rows into table collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 4d1485f0-110e-4161-9a33-c4610d68be5d n_rows = 1 table_name = measurements_i64
129042025-04-30T06:44:18.382ZTRACtest_utilization (results-sink): successfully inserted samples collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
129052025-04-30T06:44:18.450ZDEBGtest_utilization: attempt to fetch latest inventory collection result = Ok("found")
129062025-04-30T06:44:18.451ZWARNtest_utilization (datastore): Non-retryable transaction failure err = DatabaseError(UniqueViolation, "duplicate key value violates unique constraint \\"inv_caboose_pkey\\"") transaction = inventory_insert_collection
129072025-04-30T06:44:18.451ZWARNtest_utilization (BackgroundTasks): inventory collection failed background_task = inventory_collection error = failed to collect inventory: saving inventory to database: Internal Error: unexpected database error: duplicate key value violates unique constraint "inv_caboose_pkey"
129082025-04-30T06:44:18.452ZDEBGtest_utilization (BackgroundTasks): activation complete background_task = inventory_collection elapsed = 487.034856ms iteration = 33 status = {"error":"failed to collect inventory: saving inventory to database: Internal Error: unexpected database error: duplicate key value violates unique constraint \\"inv_caboose_pkey\\""}
129092025-04-30T06:44:18.452ZDEBGtest_utilization (BackgroundTasks): activating background_task = inventory_collection iteration = 34 reason = Signaled
129102025-04-30T06:44:18.452ZTRACtest_utilization (BackgroundTasks): authorize begin action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory
129112025-04-30T06:44:18.452ZTRACtest_utilization (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
129122025-04-30T06:44:18.452ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
129132025-04-30T06:44:18.452ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
129142025-04-30T06:44:18.452ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
129152025-04-30T06:44:18.458ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
129162025-04-30T06:44:18.472ZDEBGtest_utilization (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory result = Ok(())
129172025-04-30T06:44:18.472ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
129182025-04-30T06:44:18.472ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
129192025-04-30T06:44:18.473ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
129202025-04-30T06:44:18.473ZWARNtest_utilization (results-sink-cluster): failed to insert some results into metric DB collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 error = Failed to check out connection to database: No backends found for this service file = oximeter/collector/src/results_sink.rs:92
129212025-04-30T06:44:18.477ZDEBGtest_utilization (BackgroundTasks): inventory_prune_one: nothing eligible for removal (too few) background_task = inventory_collection candidates = [(13d8b8d9-6c6b-47a6-9190-952544edb75b, 0)]
129222025-04-30T06:44:18.477ZTRACtest_utilization (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal
129232025-04-30T06:44:18.478ZTRACtest_utilization (dns): buffer buffer = Length: 67 (0x43) bytes\n0000: d1 35 01 00 00 01 00 00 00 00 00 01 04 5f 6d 67 .5..........._mg\n0010: 73 04 5f 74 63 70 0d 63 6f 6e 74 72 6f 6c 2d 70 s._tcp.control-p\n0020: 6c 61 6e 65 05 6f 78 69 64 65 08 69 6e 74 65 72 lane.oxide.inter\n0030: 6e 61 6c 00 00 21 00 01 00 00 29 04 d0 00 00 00 nal..!....).....\n0040: 00 00 00 ... kind = dns peer_addr = [::1]:65477 req_id = 5b1efc4a-c3a1-4a2d-a4de-ab7b3656d6d9
129242025-04-30T06:44:18.478ZDEBGtest_utilization (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 53557,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_mgs._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_mgs._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 4,\n 95,\n 109,\n 103,\n 115,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:65477 req_id = 5b1efc4a-c3a1-4a2d-a4de-ab7b3656d6d9
129252025-04-30T06:44:18.478ZDEBGtest_utilization (store): query key key = _mgs._tcp
129262025-04-30T06:44:18.479ZDEBGtest_utilization (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
129272025-04-30T06:44:18.479ZDEBGtest_utilization (dns): dns response additional_records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal"), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] kind = dns peer_addr = [::1]:65477 query = LowerQuery { name: LowerName(Name("_mgs._tcp.control-plane.oxide.internal.")), original: Query { name: Name("_mgs._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN } } records = [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 48759, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal") })) }] req_id = 5b1efc4a-c3a1-4a2d-a4de-ab7b3656d6d9
129282025-04-30T06:44:18.479ZDEBGtest_utilization (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgs._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 48759, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }], valid_until: Instant { tv_sec: 4323, tv_nsec: 278822257 } })
129292025-04-30T06:44:18.481ZTRACtest_utilization (DnsResolver): lookup_service_targets: looking up SRV target
129302025-04-30T06:44:18.481ZTRACtest_utilization (dns): buffer buffer = Length: 108 (0x6c) bytes\n0000: a2 3c 01 00 00 01 00 00 00 00 00 01 2d 64 65 6e .<..........-den\n0010: 64 72 69 74 65 2d 62 36 64 36 35 33 34 31 2d 31 drite-b6d65341-1\n0020: 36 37 63 2d 34 31 64 66 2d 39 62 35 63 2d 34 31 67c-41df-9b5c-41\n0030: 63 64 65 64 39 39 63 32 32 39 04 68 6f 73 74 0d cded99c229.host.\n0040: 63 6f 6e 74 72 6f 6c 2d 70 6c 61 6e 65 05 6f 78 control-plane.ox\n0050: 69 64 65 08 69 6e 74 65 72 6e 61 6c 00 00 1c 00 ide.internal....\n0060: 01 00 00 29 04 d0 00 00 00 00 00 00 ...)........ kind = dns peer_addr = [::1]:54818 req_id = cf420371-723c-4ff5-bb24-a4ab4a216f1f
129312025-04-30T06:44:18.481ZDEBGtest_utilization (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 41532,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n query_type: AAAA,\n query_class: IN,\n },\n },\n original: [\n 45,\n 100,\n 101,\n 110,\n 100,\n 114,\n 105,\n 116,\n 101,\n 45,\n 98,\n 54,\n 100,\n 54,\n 53,\n 51,\n 52,\n 49,\n 45,\n 49,\n 54,\n 55,\n 99,\n 45,\n 52,\n 49,\n 100,\n 102,\n 45,\n 57,\n 98,\n 53,\n 99,\n 45,\n 52,\n 49,\n 99,\n 100,\n 101,\n 100,\n 57,\n 57,\n 99,\n 50,\n 50,\n 57,\n 4,\n 104,\n 111,\n 115,\n 116,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 28,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:54818 req_id = cf420371-723c-4ff5-bb24-a4ab4a216f1f
129322025-04-30T06:44:18.481ZDEBGtest_utilization (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
129332025-04-30T06:44:18.481ZDEBGtest_utilization (dns): dns response additional_records = [] kind = dns peer_addr = [::1]:54818 query = LowerQuery { name: LowerName(Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.")), original: Query { name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), query_type: AAAA, query_class: IN } } records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] req_id = cf420371-723c-4ff5-bb24-a4ab4a216f1f
129342025-04-30T06:44:18.533ZTRACtest_utilization (DnsResolver): lookup_all_socket_v6 srv dns_name = _clickhouse-admin-keeper._tcp.control-plane.oxide.internal
129352025-04-30T06:44:18.533ZTRACtest_utilization (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory
129362025-04-30T06:44:18.533ZTRACtest_utilization (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
129372025-04-30T06:44:18.534ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
129382025-04-30T06:44:18.534ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
129392025-04-30T06:44:18.535ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
129402025-04-30T06:44:18.535ZTRACtest_utilization (dns): buffer buffer = Length: 87 (0x57) bytes\n0000: 2f 67 01 00 00 01 00 00 00 00 00 01 18 5f 63 6c /g..........._cl\n0010: 69 63 6b 68 6f 75 73 65 2d 61 64 6d 69 6e 2d 6b ickhouse-admin-k\n0020: 65 65 70 65 72 04 5f 74 63 70 0d 63 6f 6e 74 72 eeper._tcp.contr\n0030: 6f 6c 2d 70 6c 61 6e 65 05 6f 78 69 64 65 08 69 ol-plane.oxide.i\n0040: 6e 74 65 72 6e 61 6c 00 00 21 00 01 00 00 29 04 nternal..!....).\n0050: d0 00 00 00 00 00 00 ....... kind = dns peer_addr = [::1]:55044 req_id = 1cf7b14b-0db1-407f-95ef-900551c8d436
129412025-04-30T06:44:18.535ZDEBGtest_utilization (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 12135,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 24,\n 95,\n 99,\n 108,\n 105,\n 99,\n 107,\n 104,\n 111,\n 117,\n 115,\n 101,\n 45,\n 97,\n 100,\n 109,\n 105,\n 110,\n 45,\n 107,\n 101,\n 101,\n 112,\n 101,\n 114,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:55044 req_id = 1cf7b14b-0db1-407f-95ef-900551c8d436
129422025-04-30T06:44:18.535ZDEBGtest_utilization (store): query key key = _clickhouse-admin-keeper._tcp
129432025-04-30T06:44:18.536ZERROtest_utilization (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 12135, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 1, }, query: WireQuery { query: LowerQuery { name: LowerName( Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), ), original: Query { name: Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN, }, }, original: [ 24, 95, 99, 108, 105, 99, 107, 104, 111, 117, 115, 101, 45, 97, 100, 109, 105, 110, 45, 107, 101, 101, 112, 101, 114, 4, 95, 116, 99, 112, 13, 99, 111, 110, 116, 114, 111, 108, 45, 112, 108, 97, 110, 101, 5, 111, 120, 105, 100, 101, 8, 105, 110, 116, 101, 114, 110, 97, 108, 0, 0, 33, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: Some( Edns { rcode_high: 0, version: 0, dnssec_ok: false, max_payload: 1232, options: OPT { options: {}, }, }, ), } NXDOMAIN: no records found for name: "_clickhouse-admin-keeper._tcp.control-plane.oxide.internal." kind = dns peer_addr = [::1]:55044 req_id = 1cf7b14b-0db1-407f-95ef-900551c8d436
129442025-04-30T06:44:18.536ZDEBGtest_utilization (BackgroundTasks): begin collection background_task = inventory_collection
129452025-04-30T06:44:18.536ZDEBGtest_utilization (BackgroundTasks): begin collection from MGS background_task = inventory_collection mgs_url = http://[::1]:48759
129462025-04-30T06:44:18.536ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/ignition
129472025-04-30T06:44:18.538ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
129482025-04-30T06:44:18.552ZDEBGtest_utilization (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory result = Ok(())
129492025-04-30T06:44:18.553ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
129502025-04-30T06:44:18.553ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
129512025-04-30T06:44:18.554ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
129522025-04-30T06:44:18.561ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
129532025-04-30T06:44:18.562ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
129542025-04-30T06:44:18.563ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
129552025-04-30T06:44:18.564ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/ignition", status: 200, headers: {"content-type": "application/json", "x-request-id": "6c23e654-f91d-4a2c-ac57-7243109b057b", "content-length": "835", "date": "Wed, 30 Apr 2025 06:44:17 GMT"} })
129562025-04-30T06:44:18.565ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0
129572025-04-30T06:44:18.592ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "ed3230e5-c41c-446f-85f5-dd4d8c1e0a07", "content-length": "734", "date": "Wed, 30 Apr 2025 06:44:17 GMT"} })
129582025-04-30T06:44:18.592ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0/component/sp/caboose?firmware_slot=0
129592025-04-30T06:44:18.616ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "333e6183-e6c4-42ab-916f-d61777414ccd", "content-length": "111", "date": "Wed, 30 Apr 2025 06:44:17 GMT"} })
129602025-04-30T06:44:18.616ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0/component/sp/caboose?firmware_slot=1
129612025-04-30T06:44:18.626ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "4e57f402-25f0-4385-8c03-c92e6e15f79c", "content-length": "111", "date": "Wed, 30 Apr 2025 06:44:17 GMT"} })
129622025-04-30T06:44:18.626ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0/component/rot/caboose?firmware_slot=0
129632025-04-30T06:44:18.627ZTRACtest_utilization (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal
129642025-04-30T06:44:18.627ZTRACtest_utilization (dns): buffer buffer = Length: 72 (0x48) bytes\n0000: 5e 96 01 00 00 01 00 00 00 00 00 01 09 5f 64 65 ^............_de\n0010: 6e 64 72 69 74 65 04 5f 74 63 70 0d 63 6f 6e 74 ndrite._tcp.cont\n0020: 72 6f 6c 2d 70 6c 61 6e 65 05 6f 78 69 64 65 08 rol-plane.oxide.\n0030: 69 6e 74 65 72 6e 61 6c 00 00 21 00 01 00 00 29 internal..!....)\n0040: 04 d0 00 00 00 00 00 00 ........ kind = dns peer_addr = [::1]:60253 req_id = 9c877576-7814-4b39-9d8d-5e9a1861c914
129652025-04-30T06:44:18.627ZDEBGtest_utilization (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 24214,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_dendrite._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_dendrite._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 9,\n 95,\n 100,\n 101,\n 110,\n 100,\n 114,\n 105,\n 116,\n 101,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:60253 req_id = 9c877576-7814-4b39-9d8d-5e9a1861c914
129662025-04-30T06:44:18.628ZDEBGtest_utilization (store): query key key = _dendrite._tcp
129672025-04-30T06:44:18.628ZDEBGtest_utilization (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
129682025-04-30T06:44:18.628ZDEBGtest_utilization (dns): dns response additional_records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal"), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] kind = dns peer_addr = [::1]:60253 query = LowerQuery { name: LowerName(Name("_dendrite._tcp.control-plane.oxide.internal.")), original: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN } } records = [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 47974, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal") })) }] req_id = 9c877576-7814-4b39-9d8d-5e9a1861c914
129692025-04-30T06:44:18.628ZDEBGtest_utilization (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 47974, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }], valid_until: Instant { tv_sec: 4323, tv_nsec: 427661359 } })
129702025-04-30T06:44:18.628ZTRACtest_utilization (DnsResolver): lookup_service_targets: looking up SRV target
129712025-04-30T06:44:18.631ZTRACtest_utilization (dns): buffer buffer = Length: 108 (0x6c) bytes\n0000: 65 f6 01 00 00 01 00 00 00 00 00 01 2d 64 65 6e e...........-den\n0010: 64 72 69 74 65 2d 62 36 64 36 35 33 34 31 2d 31 drite-b6d65341-1\n0020: 36 37 63 2d 34 31 64 66 2d 39 62 35 63 2d 34 31 67c-41df-9b5c-41\n0030: 63 64 65 64 39 39 63 32 32 39 04 68 6f 73 74 0d cded99c229.host.\n0040: 63 6f 6e 74 72 6f 6c 2d 70 6c 61 6e 65 05 6f 78 control-plane.ox\n0050: 69 64 65 08 69 6e 74 65 72 6e 61 6c 00 00 1c 00 ide.internal....\n0060: 01 00 00 29 04 d0 00 00 00 00 00 00 ...)........ kind = dns peer_addr = [::1]:50510 req_id = 27f1ea0f-ae14-4884-9775-6299c8939baf
129722025-04-30T06:44:18.631ZDEBGtest_utilization (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 26102,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n query_type: AAAA,\n query_class: IN,\n },\n },\n original: [\n 45,\n 100,\n 101,\n 110,\n 100,\n 114,\n 105,\n 116,\n 101,\n 45,\n 98,\n 54,\n 100,\n 54,\n 53,\n 51,\n 52,\n 49,\n 45,\n 49,\n 54,\n 55,\n 99,\n 45,\n 52,\n 49,\n 100,\n 102,\n 45,\n 57,\n 98,\n 53,\n 99,\n 45,\n 52,\n 49,\n 99,\n 100,\n 101,\n 100,\n 57,\n 57,\n 99,\n 50,\n 50,\n 57,\n 4,\n 104,\n 111,\n 115,\n 116,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 28,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:50510 req_id = 27f1ea0f-ae14-4884-9775-6299c8939baf
129732025-04-30T06:44:18.631ZDEBGtest_utilization (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
129742025-04-30T06:44:18.632ZDEBGtest_utilization (dns): dns response additional_records = [] kind = dns peer_addr = [::1]:50510 query = LowerQuery { name: LowerName(Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.")), original: Query { name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), query_type: AAAA, query_class: IN } } records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] req_id = 27f1ea0f-ae14-4884-9775-6299c8939baf
129752025-04-30T06:44:18.632ZDEBGtest_utilization: attempt to fetch latest inventory collection result = Ok("found")
129762025-04-30T06:44:18.632ZINFOtest_utilization (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_v4_garbage_collector
129772025-04-30T06:44:18.688ZINFOtest_utilization (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_v4_garbage_collector zone_address = ::1
129782025-04-30T06:44:18.688ZDEBGtest_utilization (MgsClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[::1]:12225/local/switch-id
129792025-04-30T06:44:18.689ZDEBGtest_utilization (MgsClient): client response background_task = nat_v4_garbage_collector result = Err(reqwest::Error { kind: Request, url: "http://[::1]:12225/local/switch-id", source: hyper_util::client::legacy::Error(Connect, ConnectError("tcp connect error", Os { code: 146, kind: ConnectionRefused, message: "Connection refused" })) })
129802025-04-30T06:44:18.689ZWARNtest_utilization (BackgroundTasks): failed to identify switch slot for dendrite background_task = nat_v4_garbage_collector reason = Communication Error: error sending request for url (http://[::1]:12225/local/switch-id): client error (Connect): tcp connect error: Connection refused (os error 146): Connection refused (os error 146) zone_address = ::1
129812025-04-30T06:44:18.691ZWARNtest_utilization (BackgroundTasks): Failed to map switch zone addr: Communication Error: error sending request for url (http://[::1]:12225/local/switch-id), retrying background_task = nat_v4_garbage_collector
129822025-04-30T06:44:18.692ZTRACtest_utilization (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory
129832025-04-30T06:44:18.692ZTRACtest_utilization (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
129842025-04-30T06:44:18.692ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
129852025-04-30T06:44:18.692ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
129862025-04-30T06:44:18.693ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
129872025-04-30T06:44:18.694ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "eec295a0-172e-40fe-b50d-5cd22791f820", "content-length": "170", "date": "Wed, 30 Apr 2025 06:44:17 GMT"} })
129882025-04-30T06:44:18.695ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0/component/rot/caboose?firmware_slot=1
129892025-04-30T06:44:18.697ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
129902025-04-30T06:44:18.705ZDEBGtest_utilization (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory result = Ok(())
129912025-04-30T06:44:18.732ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
129922025-04-30T06:44:18.732ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
129932025-04-30T06:44:18.733ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
129942025-04-30T06:44:18.734ZTRACtest_utilization (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal
129952025-04-30T06:44:18.736ZTRACtest_utilization (dns): buffer buffer = Length: 72 (0x48) bytes\n0000: 86 81 01 00 00 01 00 00 00 00 00 01 09 5f 64 65 ............._de\n0010: 6e 64 72 69 74 65 04 5f 74 63 70 0d 63 6f 6e 74 ndrite._tcp.cont\n0020: 72 6f 6c 2d 70 6c 61 6e 65 05 6f 78 69 64 65 08 rol-plane.oxide.\n0030: 69 6e 74 65 72 6e 61 6c 00 00 21 00 01 00 00 29 internal..!....)\n0040: 04 d0 00 00 00 00 00 00 ........ kind = dns peer_addr = [::1]:59608 req_id = 30a25a5d-9113-4a06-9808-7d3dc76cb518
129962025-04-30T06:44:18.737ZDEBGtest_utilization (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 34433,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_dendrite._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_dendrite._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 9,\n 95,\n 100,\n 101,\n 110,\n 100,\n 114,\n 105,\n 116,\n 101,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:59608 req_id = 30a25a5d-9113-4a06-9808-7d3dc76cb518
129972025-04-30T06:44:18.739ZDEBGtest_utilization (store): query key key = _dendrite._tcp
129982025-04-30T06:44:18.739ZDEBGtest_utilization (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
129992025-04-30T06:44:18.739ZDEBGtest_utilization (dns): dns response additional_records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal"), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] kind = dns peer_addr = [::1]:59608 query = LowerQuery { name: LowerName(Name("_dendrite._tcp.control-plane.oxide.internal.")), original: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN } } records = [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 47974, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal") })) }] req_id = 30a25a5d-9113-4a06-9808-7d3dc76cb518
130002025-04-30T06:44:18.739ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
130012025-04-30T06:44:18.739ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
130022025-04-30T06:44:18.741ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
130032025-04-30T06:44:18.741ZDEBGtest_utilization (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 47974, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }], valid_until: Instant { tv_sec: 4323, tv_nsec: 540521996 } })
130042025-04-30T06:44:18.741ZTRACtest_utilization (DnsResolver): lookup_service_targets: looking up SRV target
130052025-04-30T06:44:18.742ZTRACtest_utilization (dns): buffer buffer = Length: 108 (0x6c) bytes\n0000: 89 ca 01 00 00 01 00 00 00 00 00 01 2d 64 65 6e ............-den\n0010: 64 72 69 74 65 2d 62 36 64 36 35 33 34 31 2d 31 drite-b6d65341-1\n0020: 36 37 63 2d 34 31 64 66 2d 39 62 35 63 2d 34 31 67c-41df-9b5c-41\n0030: 63 64 65 64 39 39 63 32 32 39 04 68 6f 73 74 0d cded99c229.host.\n0040: 63 6f 6e 74 72 6f 6c 2d 70 6c 61 6e 65 05 6f 78 control-plane.ox\n0050: 69 64 65 08 69 6e 74 65 72 6e 61 6c 00 00 1c 00 ide.internal....\n0060: 01 00 00 29 04 d0 00 00 00 00 00 00 ...)........ kind = dns peer_addr = [::1]:50703 req_id = 148fd5ae-ca01-4547-81b1-80f3ea3c48ec
130062025-04-30T06:44:18.742ZDEBGtest_utilization (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 35274,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n query_type: AAAA,\n query_class: IN,\n },\n },\n original: [\n 45,\n 100,\n 101,\n 110,\n 100,\n 114,\n 105,\n 116,\n 101,\n 45,\n 98,\n 54,\n 100,\n 54,\n 53,\n 51,\n 52,\n 49,\n 45,\n 49,\n 54,\n 55,\n 99,\n 45,\n 52,\n 49,\n 100,\n 102,\n 45,\n 57,\n 98,\n 53,\n 99,\n 45,\n 52,\n 49,\n 99,\n 100,\n 101,\n 100,\n 57,\n 57,\n 99,\n 50,\n 50,\n 57,\n 4,\n 104,\n 111,\n 115,\n 116,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 28,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:50703 req_id = 148fd5ae-ca01-4547-81b1-80f3ea3c48ec
130072025-04-30T06:44:18.742ZDEBGtest_utilization (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
130082025-04-30T06:44:18.743ZDEBGtest_utilization (dns): dns response additional_records = [] kind = dns peer_addr = [::1]:50703 query = LowerQuery { name: LowerName(Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.")), original: Query { name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), query_type: AAAA, query_class: IN } } records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] req_id = 148fd5ae-ca01-4547-81b1-80f3ea3c48ec
130092025-04-30T06:44:18.743ZINFOtest_utilization (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager
130102025-04-30T06:44:18.854ZINFOtest_utilization (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager zone_address = ::1
130112025-04-30T06:44:18.854ZDEBGtest_utilization (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[::1]:12225/local/switch-id
130122025-04-30T06:44:18.854ZDEBGtest_utilization (MgsClient): client response background_task = bfd_manager result = Err(reqwest::Error { kind: Request, url: "http://[::1]:12225/local/switch-id", source: hyper_util::client::legacy::Error(Connect, ConnectError("tcp connect error", Os { code: 146, kind: ConnectionRefused, message: "Connection refused" })) })
130132025-04-30T06:44:18.854ZWARNtest_utilization (BackgroundTasks): failed to identify switch slot for dendrite background_task = bfd_manager reason = Communication Error: error sending request for url (http://[::1]:12225/local/switch-id): client error (Connect): tcp connect error: Connection refused (os error 146): Connection refused (os error 146) zone_address = ::1
130142025-04-30T06:44:18.856ZWARNtest_utilization (BackgroundTasks): Failed to map switch zone addr: Communication Error: error sending request for url (http://[::1]:12225/local/switch-id), retrying background_task = bfd_manager
130152025-04-30T06:44:18.857ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "c815a351-727c-40a0-b3a9-285e6f296536", "content-length": "170", "date": "Wed, 30 Apr 2025 06:44:17 GMT"} })
130162025-04-30T06:44:18.857ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0/component/stage0/caboose?firmware_slot=0
130172025-04-30T06:44:18.862ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "7e0a5011-8151-4a12-8929-b8861e539810", "content-length": "179", "date": "Wed, 30 Apr 2025 06:44:17 GMT"} })
130182025-04-30T06:44:18.862ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0/component/stage0/caboose?firmware_slot=1
130192025-04-30T06:44:18.865ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "39e93b19-87ca-4af3-ba7f-b440f1cbf7ef", "content-length": "179", "date": "Wed, 30 Apr 2025 06:44:17 GMT"} })
130202025-04-30T06:44:18.866ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0/component/rot/cmpa
130212025-04-30T06:44:18.867ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "958e9051-bf7d-47ee-bd7b-279486670fad", "content-length": "702", "date": "Wed, 30 Apr 2025 06:44:17 GMT"} })
130222025-04-30T06:44:18.867ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0/component/rot/cfpa
130232025-04-30T06:44:18.869ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "423bab03-c673-49a7-9a40-bc779369acdf", "content-length": "727", "date": "Wed, 30 Apr 2025 06:44:17 GMT"} })
130242025-04-30T06:44:18.869ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0/component/rot/cfpa
130252025-04-30T06:44:18.870ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "de58e5fd-db6e-485a-8b1f-1375c7444957", "content-length": "729", "date": "Wed, 30 Apr 2025 06:44:17 GMT"} })
130262025-04-30T06:44:18.871ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0/component/rot/cfpa
130272025-04-30T06:44:18.872ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "0b6fbf54-a606-4806-a78a-846fe04797c9", "content-length": "728", "date": "Wed, 30 Apr 2025 06:44:17 GMT"} })
130282025-04-30T06:44:18.872ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1
130292025-04-30T06:44:18.875ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1", status: 200, headers: {"content-type": "application/json", "x-request-id": "29fd2d66-1442-492b-9041-264e051c2c4e", "content-length": "734", "date": "Wed, 30 Apr 2025 06:44:17 GMT"} })
130302025-04-30T06:44:18.875ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1/component/sp/caboose?firmware_slot=0
130312025-04-30T06:44:18.881ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "73196c84-db4b-425c-9764-3aeef7511216", "content-length": "111", "date": "Wed, 30 Apr 2025 06:44:17 GMT"} })
130322025-04-30T06:44:18.882ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1/component/sp/caboose?firmware_slot=1
130332025-04-30T06:44:18.892ZDEBGtest_utilization: attempt to fetch latest inventory collection result = Ok("found")
130342025-04-30T06:44:18.896ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "98fa62ea-ae53-4af4-a053-49859bd83b0c", "content-length": "111", "date": "Wed, 30 Apr 2025 06:44:17 GMT"} })
130352025-04-30T06:44:18.896ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1/component/rot/caboose?firmware_slot=0
130362025-04-30T06:44:18.900ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "41ccf58c-36f9-401f-9b0b-546003e1e761", "content-length": "170", "date": "Wed, 30 Apr 2025 06:44:17 GMT"} })
130372025-04-30T06:44:18.900ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1/component/rot/caboose?firmware_slot=1
130382025-04-30T06:44:18.903ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "cda20312-f73c-4ea4-8de3-96b1d8828064", "content-length": "170", "date": "Wed, 30 Apr 2025 06:44:17 GMT"} })
130392025-04-30T06:44:18.904ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1/component/stage0/caboose?firmware_slot=0
130402025-04-30T06:44:18.908ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "9392646b-9d27-4461-99e0-fbfce4ffdabc", "content-length": "179", "date": "Wed, 30 Apr 2025 06:44:17 GMT"} })
130412025-04-30T06:44:18.908ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1/component/stage0/caboose?firmware_slot=1
130422025-04-30T06:44:18.910ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "f20d89ab-b2a0-4aa3-a10c-3902cd7045ca", "content-length": "179", "date": "Wed, 30 Apr 2025 06:44:17 GMT"} })
130432025-04-30T06:44:18.910ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1/component/rot/cmpa
130442025-04-30T06:44:18.911ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "9181e9bc-32fd-43c0-b984-0334835a0fb3", "content-length": "702", "date": "Wed, 30 Apr 2025 06:44:17 GMT"} })
130452025-04-30T06:44:18.911ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1/component/rot/cfpa
130462025-04-30T06:44:18.912ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "3450dc09-dcf2-40b5-b3ff-f076e12c0c32", "content-length": "727", "date": "Wed, 30 Apr 2025 06:44:17 GMT"} })
130472025-04-30T06:44:18.912ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1/component/rot/cfpa
130482025-04-30T06:44:18.917ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "426b3477-7419-4951-a69b-8ec93e363802", "content-length": "729", "date": "Wed, 30 Apr 2025 06:44:18 GMT"} })
130492025-04-30T06:44:18.917ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1/component/rot/cfpa
130502025-04-30T06:44:18.918ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "5114de52-e535-4eae-8fec-5f2cf19afc2f", "content-length": "728", "date": "Wed, 30 Apr 2025 06:44:18 GMT"} })
130512025-04-30T06:44:18.918ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0
130522025-04-30T06:44:18.920ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "50938aec-95d7-4a1b-ba9d-bcfa6870adcc", "content-length": "723", "date": "Wed, 30 Apr 2025 06:44:18 GMT"} })
130532025-04-30T06:44:18.920ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0/component/sp/caboose?firmware_slot=0
130542025-04-30T06:44:18.925ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "bc746d4c-6938-4c78-87d9-2d42781b336b", "content-length": "109", "date": "Wed, 30 Apr 2025 06:44:18 GMT"} })
130552025-04-30T06:44:18.927ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0/component/sp/caboose?firmware_slot=1
130562025-04-30T06:44:18.931ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "f72e55d6-678e-4eeb-889f-9f512da1c722", "content-length": "109", "date": "Wed, 30 Apr 2025 06:44:18 GMT"} })
130572025-04-30T06:44:18.932ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0/component/rot/caboose?firmware_slot=0
130582025-04-30T06:44:18.937ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "81c88f8d-2600-407f-835f-dc8b92454017", "content-length": "169", "date": "Wed, 30 Apr 2025 06:44:18 GMT"} })
130592025-04-30T06:44:18.938ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0/component/rot/caboose?firmware_slot=1
130602025-04-30T06:44:18.940ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "3682a8dc-5bab-4573-8637-4eff5284c654", "content-length": "169", "date": "Wed, 30 Apr 2025 06:44:18 GMT"} })
130612025-04-30T06:44:18.940ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0/component/stage0/caboose?firmware_slot=0
130622025-04-30T06:44:18.942ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "b5f158d0-3819-4db0-9f6f-4b2c449dad25", "content-length": "178", "date": "Wed, 30 Apr 2025 06:44:18 GMT"} })
130632025-04-30T06:44:18.942ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0/component/stage0/caboose?firmware_slot=1
130642025-04-30T06:44:18.943ZTRACtest_utilization (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory
130652025-04-30T06:44:18.943ZTRACtest_utilization (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
130662025-04-30T06:44:18.943ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
130672025-04-30T06:44:18.943ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
130682025-04-30T06:44:18.944ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
130692025-04-30T06:44:18.946ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "f64510cd-572a-4c2e-83a0-7c90579b8a7f", "content-length": "178", "date": "Wed, 30 Apr 2025 06:44:18 GMT"} })
130702025-04-30T06:44:18.946ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0/component/rot/cmpa
130712025-04-30T06:44:18.947ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
130722025-04-30T06:44:18.953ZDEBGtest_utilization (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory result = Ok(())
130732025-04-30T06:44:18.953ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
130742025-04-30T06:44:18.953ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
130752025-04-30T06:44:18.953ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
130762025-04-30T06:44:18.954ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "a1526105-3e78-4f5c-829b-cc2470814f3f", "content-length": "702", "date": "Wed, 30 Apr 2025 06:44:18 GMT"} })
130772025-04-30T06:44:18.954ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0/component/rot/cfpa
130782025-04-30T06:44:18.956ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "d7159816-b4c8-48ed-872d-1828c3b4b704", "content-length": "727", "date": "Wed, 30 Apr 2025 06:44:18 GMT"} })
130792025-04-30T06:44:18.956ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0/component/rot/cfpa
130802025-04-30T06:44:18.957ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
130812025-04-30T06:44:18.957ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
130822025-04-30T06:44:18.957ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
130832025-04-30T06:44:18.959ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "fa174ba3-3fd5-45b1-9970-e99ce9498fcd", "content-length": "729", "date": "Wed, 30 Apr 2025 06:44:18 GMT"} })
130842025-04-30T06:44:18.959ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0/component/rot/cfpa
130852025-04-30T06:44:18.961ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "a2ee843e-91a7-41b1-8d2c-c4864a907339", "content-length": "728", "date": "Wed, 30 Apr 2025 06:44:18 GMT"} })
130862025-04-30T06:44:18.961ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1
130872025-04-30T06:44:18.964ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1", status: 200, headers: {"content-type": "application/json", "x-request-id": "9d5d625c-029c-4e0f-b904-50f37b08fe39", "content-length": "723", "date": "Wed, 30 Apr 2025 06:44:18 GMT"} })
130882025-04-30T06:44:18.964ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1/component/sp/caboose?firmware_slot=0
130892025-04-30T06:44:18.968ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "749174be-22b2-49e4-b730-dd0235ea8ac2", "content-length": "109", "date": "Wed, 30 Apr 2025 06:44:18 GMT"} })
130902025-04-30T06:44:18.968ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1/component/sp/caboose?firmware_slot=1
130912025-04-30T06:44:18.970ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "56fbf01f-cfcd-454f-bd87-9e2d2d50bf86", "content-length": "109", "date": "Wed, 30 Apr 2025 06:44:18 GMT"} })
130922025-04-30T06:44:18.971ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1/component/rot/caboose?firmware_slot=0
130932025-04-30T06:44:18.975ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "dd4f2e27-4c59-45dc-ad30-f5dc1002e753", "content-length": "169", "date": "Wed, 30 Apr 2025 06:44:18 GMT"} })
130942025-04-30T06:44:18.975ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1/component/rot/caboose?firmware_slot=1
130952025-04-30T06:44:18.977ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "a8311fb5-6ccc-48fa-96d6-897134d53281", "content-length": "169", "date": "Wed, 30 Apr 2025 06:44:18 GMT"} })
130962025-04-30T06:44:18.977ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1/component/stage0/caboose?firmware_slot=0
130972025-04-30T06:44:18.980ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "10851711-9af8-41d2-940a-00c0d349ef06", "content-length": "178", "date": "Wed, 30 Apr 2025 06:44:18 GMT"} })
130982025-04-30T06:44:18.980ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1/component/stage0/caboose?firmware_slot=1
130992025-04-30T06:44:18.984ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "76c1c2f6-9013-40ba-8317-49f9391e64de", "content-length": "178", "date": "Wed, 30 Apr 2025 06:44:18 GMT"} })
131002025-04-30T06:44:18.984ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1/component/rot/cmpa
131012025-04-30T06:44:18.985ZDEBGtest_utilization: attempt to fetch latest inventory collection result = Ok("found")
131022025-04-30T06:44:18.987ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "85de0edc-472c-45db-a17c-de6cd94b22cd", "content-length": "702", "date": "Wed, 30 Apr 2025 06:44:18 GMT"} })
131032025-04-30T06:44:18.988ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1/component/rot/cfpa
131042025-04-30T06:44:18.990ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "2fa85b7f-e983-4daa-bdb9-5ffd24538e5f", "content-length": "727", "date": "Wed, 30 Apr 2025 06:44:18 GMT"} })
131052025-04-30T06:44:18.990ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1/component/rot/cfpa
131062025-04-30T06:44:18.991ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "2f0958df-d171-40f7-885e-57e9b2f90012", "content-length": "729", "date": "Wed, 30 Apr 2025 06:44:18 GMT"} })
131072025-04-30T06:44:18.992ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1/component/rot/cfpa
131082025-04-30T06:44:18.993ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "d160a654-c046-48c3-87e9-4dae132b5190", "content-length": "728", "date": "Wed, 30 Apr 2025 06:44:18 GMT"} })
131092025-04-30T06:44:18.993ZTRACtest_utilization (BackgroundTasks): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet
131102025-04-30T06:44:18.994ZTRACtest_utilization (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
131112025-04-30T06:44:18.994ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
131122025-04-30T06:44:18.994ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
131132025-04-30T06:44:18.995ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
131142025-04-30T06:44:18.996ZDEBGtest_utilization (collection-task): sent timer-based collection request to the collection task collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
131152025-04-30T06:44:18.996ZDEBGtest_utilization (collection-task): collecting from producer collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
131162025-04-30T06:44:18.998ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
131172025-04-30T06:44:19.008ZDEBGtest_utilization (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet result = Ok(())
131182025-04-30T06:44:19.008ZTRACtest_utilization (BackgroundTasks): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet
131192025-04-30T06:44:19.008ZTRACtest_utilization (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
131202025-04-30T06:44:19.008ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
131212025-04-30T06:44:19.008ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
131222025-04-30T06:44:19.009ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
131232025-04-30T06:44:19.010ZTRACtest_utilization (collection-task): sent collection request to producer collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
131242025-04-30T06:44:19.011ZDEBGtest_utilization (collection-task): collected results from producer collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 n_results = 1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
131252025-04-30T06:44:19.011ZTRACtest_utilization (collection-task): forwarded results to main collection loop collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
131262025-04-30T06:44:19.011ZTRACtest_utilization (collection-task): top of inner collection loop, waiting for next request collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
131272025-04-30T06:44:19.012ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
131282025-04-30T06:44:19.020ZDEBGtest_utilization (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet result = Ok(())
131292025-04-30T06:44:19.020ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
131302025-04-30T06:44:19.020ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
131312025-04-30T06:44:19.021ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
131322025-04-30T06:44:19.023ZTRACtest_utilization (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal
131332025-04-30T06:44:19.023ZTRACtest_utilization (dns): buffer buffer = Length: 72 (0x48) bytes\n0000: 69 af 01 00 00 01 00 00 00 00 00 01 09 5f 64 65 i............_de\n0010: 6e 64 72 69 74 65 04 5f 74 63 70 0d 63 6f 6e 74 ndrite._tcp.cont\n0020: 72 6f 6c 2d 70 6c 61 6e 65 05 6f 78 69 64 65 08 rol-plane.oxide.\n0030: 69 6e 74 65 72 6e 61 6c 00 00 21 00 01 00 00 29 internal..!....)\n0040: 04 d0 00 00 00 00 00 00 ........ kind = dns peer_addr = [::1]:55887 req_id = 2737d389-3153-41c7-bff5-5ff009a5637a
131342025-04-30T06:44:19.023ZDEBGtest_utilization (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 27055,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_dendrite._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_dendrite._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 9,\n 95,\n 100,\n 101,\n 110,\n 100,\n 114,\n 105,\n 116,\n 101,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:55887 req_id = 2737d389-3153-41c7-bff5-5ff009a5637a
131352025-04-30T06:44:19.023ZDEBGtest_utilization (store): query key key = _dendrite._tcp
131362025-04-30T06:44:19.024ZDEBGtest_utilization (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
131372025-04-30T06:44:19.024ZDEBGtest_utilization (dns): dns response additional_records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal"), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] kind = dns peer_addr = [::1]:55887 query = LowerQuery { name: LowerName(Name("_dendrite._tcp.control-plane.oxide.internal.")), original: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN } } records = [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 47974, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal") })) }] req_id = 2737d389-3153-41c7-bff5-5ff009a5637a
131382025-04-30T06:44:19.024ZDEBGtest_utilization (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 47974, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }], valid_until: Instant { tv_sec: 4323, tv_nsec: 823433669 } })
131392025-04-30T06:44:19.024ZTRACtest_utilization (DnsResolver): lookup_service_targets: looking up SRV target
131402025-04-30T06:44:19.024ZTRACtest_utilization (dns): buffer buffer = Length: 108 (0x6c) bytes\n0000: 23 9a 01 00 00 01 00 00 00 00 00 01 2d 64 65 6e #...........-den\n0010: 64 72 69 74 65 2d 62 36 64 36 35 33 34 31 2d 31 drite-b6d65341-1\n0020: 36 37 63 2d 34 31 64 66 2d 39 62 35 63 2d 34 31 67c-41df-9b5c-41\n0030: 63 64 65 64 39 39 63 32 32 39 04 68 6f 73 74 0d cded99c229.host.\n0040: 63 6f 6e 74 72 6f 6c 2d 70 6c 61 6e 65 05 6f 78 control-plane.ox\n0050: 69 64 65 08 69 6e 74 65 72 6e 61 6c 00 00 1c 00 ide.internal....\n0060: 01 00 00 29 04 d0 00 00 00 00 00 00 ...)........ kind = dns peer_addr = [::1]:52588 req_id = 10aa9f13-9999-4ca9-8e63-24e91b09f9e9
131412025-04-30T06:44:19.025ZDEBGtest_utilization (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 9114,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n query_type: AAAA,\n query_class: IN,\n },\n },\n original: [\n 45,\n 100,\n 101,\n 110,\n 100,\n 114,\n 105,\n 116,\n 101,\n 45,\n 98,\n 54,\n 100,\n 54,\n 53,\n 51,\n 52,\n 49,\n 45,\n 49,\n 54,\n 55,\n 99,\n 45,\n 52,\n 49,\n 100,\n 102,\n 45,\n 57,\n 98,\n 53,\n 99,\n 45,\n 52,\n 49,\n 99,\n 100,\n 101,\n 100,\n 57,\n 57,\n 99,\n 50,\n 50,\n 57,\n 4,\n 104,\n 111,\n 115,\n 116,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 28,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:52588 req_id = 10aa9f13-9999-4ca9-8e63-24e91b09f9e9
131422025-04-30T06:44:19.025ZDEBGtest_utilization (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
131432025-04-30T06:44:19.025ZDEBGtest_utilization (dns): dns response additional_records = [] kind = dns peer_addr = [::1]:52588 query = LowerQuery { name: LowerName(Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.")), original: Query { name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), query_type: AAAA, query_class: IN } } records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] req_id = 10aa9f13-9999-4ca9-8e63-24e91b09f9e9
131442025-04-30T06:44:19.025ZINFOtest_utilization (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager rack_id = c19a698f-c6f9-4a17-ae30-20d711b8f7dc
131452025-04-30T06:44:19.139ZINFOtest_utilization (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager rack_id = c19a698f-c6f9-4a17-ae30-20d711b8f7dc zone_address = ::1
131462025-04-30T06:44:19.139ZDEBGtest_utilization (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = c19a698f-c6f9-4a17-ae30-20d711b8f7dc uri = http://[::1]:12225/local/switch-id
131472025-04-30T06:44:19.139ZDEBGtest_utilization (MgsClient): client response background_task = switch_port_config_manager rack_id = c19a698f-c6f9-4a17-ae30-20d711b8f7dc result = Err(reqwest::Error { kind: Request, url: "http://[::1]:12225/local/switch-id", source: hyper_util::client::legacy::Error(Connect, ConnectError("tcp connect error", Os { code: 146, kind: ConnectionRefused, message: "Connection refused" })) })
131482025-04-30T06:44:19.139ZWARNtest_utilization (BackgroundTasks): failed to identify switch slot for dendrite background_task = switch_port_config_manager rack_id = c19a698f-c6f9-4a17-ae30-20d711b8f7dc reason = Communication Error: error sending request for url (http://[::1]:12225/local/switch-id): client error (Connect): tcp connect error: Connection refused (os error 146): Connection refused (os error 146) zone_address = ::1
131492025-04-30T06:44:19.141ZWARNtest_utilization (BackgroundTasks): Failed to map switch zone addr: Communication Error: error sending request for url (http://[::1]:12225/local/switch-id), retrying background_task = switch_port_config_manager rack_id = c19a698f-c6f9-4a17-ae30-20d711b8f7dc
131502025-04-30T06:44:19.207ZDEBGtest_utilization (BackgroundTasks): begin collection from Sled Agent background_task = inventory_collection sled_agent_url = http://[::1]:39365
131512025-04-30T06:44:19.207ZDEBGtest_utilization (BackgroundTasks): client request SledAgent = http://[::1]:39365 background_task = inventory_collection body = None method = GET uri = http://[::1]:39365/inventory
131522025-04-30T06:44:19.208ZTRACtest_utilization (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory
131532025-04-30T06:44:19.208ZTRACtest_utilization (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
131542025-04-30T06:44:19.208ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
131552025-04-30T06:44:19.208ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
131562025-04-30T06:44:19.209ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
131572025-04-30T06:44:19.209ZINFOtest_utilization (dropshot): accepted connection local_addr = [::1]:39365 remote_addr = [::1]:38621 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
131582025-04-30T06:44:19.212ZTRACtest_utilization (dropshot): incoming request local_addr = [::1]:39365 method = GET remote_addr = [::1]:38621 req_id = 864a7222-69ca-463f-ba76-1fd93a337716 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /inventory
131592025-04-30T06:44:19.213ZINFOtest_utilization (dropshot): request completed latency_us = 1506 local_addr = [::1]:39365 method = GET remote_addr = [::1]:38621 req_id = 864a7222-69ca-463f-ba76-1fd93a337716 response_code = 200 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /inventory
131602025-04-30T06:44:19.213ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
131612025-04-30T06:44:19.222ZDEBGtest_utilization (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory result = Ok(())
131622025-04-30T06:44:19.222ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
131632025-04-30T06:44:19.222ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
131642025-04-30T06:44:19.223ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
131652025-04-30T06:44:19.223ZDEBGtest_utilization (BackgroundTasks): client response SledAgent = http://[::1]:39365 background_task = inventory_collection result = Ok(Response { url: "http://[::1]:39365/inventory", status: 200, headers: {"content-type": "application/json", "x-request-id": "864a7222-69ca-463f-ba76-1fd93a337716", "content-length": "3141", "date": "Wed, 30 Apr 2025 06:44:18 GMT"} })
131662025-04-30T06:44:19.226ZDEBGtest_utilization (BackgroundTasks): begin collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
131672025-04-30T06:44:19.226ZDEBGtest_utilization (BackgroundTasks): end collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
131682025-04-30T06:44:19.226ZDEBGtest_utilization (BackgroundTasks): finished collection background_task = inventory_collection
131692025-04-30T06:44:19.230ZTRACtest_utilization (BackgroundTasks): authorize begin action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory
131702025-04-30T06:44:19.230ZTRACtest_utilization (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
131712025-04-30T06:44:19.230ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
131722025-04-30T06:44:19.230ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
131732025-04-30T06:44:19.231ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
131742025-04-30T06:44:19.241ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
131752025-04-30T06:44:19.241ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
131762025-04-30T06:44:19.242ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
131772025-04-30T06:44:19.244ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
131782025-04-30T06:44:19.248ZDEBGtest_utilization (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory result = Ok(())
131792025-04-30T06:44:19.248ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
131802025-04-30T06:44:19.248ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
131812025-04-30T06:44:19.250ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
131822025-04-30T06:44:19.284ZWARNtest_utilization (datastore): Non-retryable transaction failure err = DatabaseError(UniqueViolation, "duplicate key value violates unique constraint \\"inv_caboose_pkey\\"") transaction = inventory_insert_collection
131832025-04-30T06:44:19.285ZWARNtest_utilization (BackgroundTasks): inventory collection failed background_task = inventory_collection error = failed to collect inventory: saving inventory to database: Internal Error: unexpected database error: duplicate key value violates unique constraint "inv_caboose_pkey"
131842025-04-30T06:44:19.285ZDEBGtest_utilization (BackgroundTasks): activation complete background_task = inventory_collection elapsed = 833.063173ms iteration = 34 status = {"error":"failed to collect inventory: saving inventory to database: Internal Error: unexpected database error: duplicate key value violates unique constraint \\"inv_caboose_pkey\\""}
131852025-04-30T06:44:19.285ZDEBGtest_utilization (BackgroundTasks): activating background_task = inventory_collection iteration = 35 reason = Signaled
131862025-04-30T06:44:19.285ZTRACtest_utilization (BackgroundTasks): authorize begin action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory
131872025-04-30T06:44:19.285ZTRACtest_utilization (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
131882025-04-30T06:44:19.285ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
131892025-04-30T06:44:19.285ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
131902025-04-30T06:44:19.286ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
131912025-04-30T06:44:19.289ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
131922025-04-30T06:44:19.307ZDEBGtest_utilization (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory result = Ok(())
131932025-04-30T06:44:19.308ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
131942025-04-30T06:44:19.308ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
131952025-04-30T06:44:19.308ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
131962025-04-30T06:44:19.315ZDEBGtest_utilization (BackgroundTasks): inventory_prune_one: nothing eligible for removal (too few) background_task = inventory_collection candidates = [(13d8b8d9-6c6b-47a6-9190-952544edb75b, 0)]
131972025-04-30T06:44:19.315ZTRACtest_utilization (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal
131982025-04-30T06:44:19.315ZTRACtest_utilization (dns): buffer buffer = Length: 67 (0x43) bytes\n0000: b8 f3 01 00 00 01 00 00 00 00 00 01 04 5f 6d 67 ............._mg\n0010: 73 04 5f 74 63 70 0d 63 6f 6e 74 72 6f 6c 2d 70 s._tcp.control-p\n0020: 6c 61 6e 65 05 6f 78 69 64 65 08 69 6e 74 65 72 lane.oxide.inter\n0030: 6e 61 6c 00 00 21 00 01 00 00 29 04 d0 00 00 00 nal..!....).....\n0040: 00 00 00 ... kind = dns peer_addr = [::1]:54488 req_id = b7159775-f07e-490e-ba48-4362a11f296e
131992025-04-30T06:44:19.315ZDEBGtest_utilization (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 47347,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_mgs._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_mgs._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 4,\n 95,\n 109,\n 103,\n 115,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:54488 req_id = b7159775-f07e-490e-ba48-4362a11f296e
132002025-04-30T06:44:19.316ZDEBGtest_utilization (store): query key key = _mgs._tcp
132012025-04-30T06:44:19.316ZDEBGtest_utilization (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
132022025-04-30T06:44:19.316ZDEBGtest_utilization (dns): dns response additional_records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal"), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] kind = dns peer_addr = [::1]:54488 query = LowerQuery { name: LowerName(Name("_mgs._tcp.control-plane.oxide.internal.")), original: Query { name: Name("_mgs._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN } } records = [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 48759, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal") })) }] req_id = b7159775-f07e-490e-ba48-4362a11f296e
132032025-04-30T06:44:19.316ZDEBGtest_utilization: attempt to fetch latest inventory collection result = Ok("found")
132042025-04-30T06:44:19.316ZDEBGtest_utilization (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgs._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 48759, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }], valid_until: Instant { tv_sec: 4324, tv_nsec: 115882414 } })
132052025-04-30T06:44:19.317ZTRACtest_utilization (DnsResolver): lookup_service_targets: looking up SRV target
132062025-04-30T06:44:19.317ZTRACtest_utilization (dns): buffer buffer = Length: 108 (0x6c) bytes\n0000: e3 b4 01 00 00 01 00 00 00 00 00 01 2d 64 65 6e ............-den\n0010: 64 72 69 74 65 2d 62 36 64 36 35 33 34 31 2d 31 drite-b6d65341-1\n0020: 36 37 63 2d 34 31 64 66 2d 39 62 35 63 2d 34 31 67c-41df-9b5c-41\n0030: 63 64 65 64 39 39 63 32 32 39 04 68 6f 73 74 0d cded99c229.host.\n0040: 63 6f 6e 74 72 6f 6c 2d 70 6c 61 6e 65 05 6f 78 control-plane.ox\n0050: 69 64 65 08 69 6e 74 65 72 6e 61 6c 00 00 1c 00 ide.internal....\n0060: 01 00 00 29 04 d0 00 00 00 00 00 00 ...)........ kind = dns peer_addr = [::1]:55702 req_id = 9fafffb8-7c05-4f52-82c2-fc88654c1768
132072025-04-30T06:44:19.317ZDEBGtest_utilization (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 58292,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n query_type: AAAA,\n query_class: IN,\n },\n },\n original: [\n 45,\n 100,\n 101,\n 110,\n 100,\n 114,\n 105,\n 116,\n 101,\n 45,\n 98,\n 54,\n 100,\n 54,\n 53,\n 51,\n 52,\n 49,\n 45,\n 49,\n 54,\n 55,\n 99,\n 45,\n 52,\n 49,\n 100,\n 102,\n 45,\n 57,\n 98,\n 53,\n 99,\n 45,\n 52,\n 49,\n 99,\n 100,\n 101,\n 100,\n 57,\n 57,\n 99,\n 50,\n 50,\n 57,\n 4,\n 104,\n 111,\n 115,\n 116,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 28,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:55702 req_id = 9fafffb8-7c05-4f52-82c2-fc88654c1768
132082025-04-30T06:44:19.317ZDEBGtest_utilization (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
132092025-04-30T06:44:19.317ZDEBGtest_utilization (dns): dns response additional_records = [] kind = dns peer_addr = [::1]:55702 query = LowerQuery { name: LowerName(Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.")), original: Query { name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), query_type: AAAA, query_class: IN } } records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] req_id = 9fafffb8-7c05-4f52-82c2-fc88654c1768
132102025-04-30T06:44:19.391ZTRACtest_utilization (DnsResolver): lookup_all_socket_v6 srv dns_name = _clickhouse-admin-keeper._tcp.control-plane.oxide.internal
132112025-04-30T06:44:19.392ZTRACtest_utilization (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory
132122025-04-30T06:44:19.392ZTRACtest_utilization (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
132132025-04-30T06:44:19.392ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
132142025-04-30T06:44:19.392ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
132152025-04-30T06:44:19.393ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
132162025-04-30T06:44:19.393ZDEBGtest_utilization (results-sink): inserting 1 samples into database collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
132172025-04-30T06:44:19.393ZDEBGtest_utilization (results-sink-cluster): inserting 1 samples into database collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
132182025-04-30T06:44:19.393ZDEBGtest_utilization (clickhouse-client): unrolling 1 total samples collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 4d1485f0-110e-4161-9a33-c4610d68be5d
132192025-04-30T06:44:19.395ZTRACtest_utilization (clickhouse-client): inserting data collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 4d1485f0-110e-4161-9a33-c4610d68be5d n_columns = 4 n_rows = 2 sql = INSERT INTO oximeter.fields_string FORMAT Native
132202025-04-30T06:44:19.395ZTRACtest_utilization (dns): buffer buffer = Length: 87 (0x57) bytes\n0000: d7 0c 01 00 00 01 00 00 00 00 00 01 18 5f 63 6c ............._cl\n0010: 69 63 6b 68 6f 75 73 65 2d 61 64 6d 69 6e 2d 6b ickhouse-admin-k\n0020: 65 65 70 65 72 04 5f 74 63 70 0d 63 6f 6e 74 72 eeper._tcp.contr\n0030: 6f 6c 2d 70 6c 61 6e 65 05 6f 78 69 64 65 08 69 ol-plane.oxide.i\n0040: 6e 74 65 72 6e 61 6c 00 00 21 00 01 00 00 29 04 nternal..!....).\n0050: d0 00 00 00 00 00 00 ....... kind = dns peer_addr = [::1]:64494 req_id = 7d1cb921-6a2a-4b25-87fd-d48f69b489f5
132212025-04-30T06:44:19.395ZDEBGtest_utilization (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 55052,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 24,\n 95,\n 99,\n 108,\n 105,\n 99,\n 107,\n 104,\n 111,\n 117,\n 115,\n 101,\n 45,\n 97,\n 100,\n 109,\n 105,\n 110,\n 45,\n 107,\n 101,\n 101,\n 112,\n 101,\n 114,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:64494 req_id = 7d1cb921-6a2a-4b25-87fd-d48f69b489f5
132222025-04-30T06:44:19.395ZDEBGtest_utilization (store): query key key = _clickhouse-admin-keeper._tcp
132232025-04-30T06:44:19.395ZERROtest_utilization (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 55052, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 1, }, query: WireQuery { query: LowerQuery { name: LowerName( Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), ), original: Query { name: Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN, }, }, original: [ 24, 95, 99, 108, 105, 99, 107, 104, 111, 117, 115, 101, 45, 97, 100, 109, 105, 110, 45, 107, 101, 101, 112, 101, 114, 4, 95, 116, 99, 112, 13, 99, 111, 110, 116, 114, 111, 108, 45, 112, 108, 97, 110, 101, 5, 111, 120, 105, 100, 101, 8, 105, 110, 116, 101, 114, 110, 97, 108, 0, 0, 33, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: Some( Edns { rcode_high: 0, version: 0, dnssec_ok: false, max_payload: 1232, options: OPT { options: {}, }, }, ), } NXDOMAIN: no records found for name: "_clickhouse-admin-keeper._tcp.control-plane.oxide.internal." kind = dns peer_addr = [::1]:64494 req_id = 7d1cb921-6a2a-4b25-87fd-d48f69b489f5
132242025-04-30T06:44:19.396ZDEBGtest_utilization (BackgroundTasks): begin collection background_task = inventory_collection
132252025-04-30T06:44:19.396ZDEBGtest_utilization (BackgroundTasks): begin collection from MGS background_task = inventory_collection mgs_url = http://[::1]:48759
132262025-04-30T06:44:19.396ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/ignition
132272025-04-30T06:44:19.397ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
132282025-04-30T06:44:19.403ZDEBGtest_utilization (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory result = Ok(())
132292025-04-30T06:44:19.403ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
132302025-04-30T06:44:19.403ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
132312025-04-30T06:44:19.403ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
132322025-04-30T06:44:19.405ZDEBGtest_utilization (clickhouse-client): inserted rows into table collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 4d1485f0-110e-4161-9a33-c4610d68be5d n_rows = 2 table_name = fields_string
132332025-04-30T06:44:19.405ZTRACtest_utilization (clickhouse-client): inserting data collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 4d1485f0-110e-4161-9a33-c4610d68be5d n_columns = 4 n_rows = 1 sql = INSERT INTO oximeter.measurements_i64 FORMAT Native
132342025-04-30T06:44:19.406ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/ignition", status: 200, headers: {"content-type": "application/json", "x-request-id": "0d73bd38-8527-432b-ba15-14987645cdc2", "content-length": "835", "date": "Wed, 30 Apr 2025 06:44:18 GMT"} })
132352025-04-30T06:44:19.406ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0
132362025-04-30T06:44:19.406ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
132372025-04-30T06:44:19.406ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
132382025-04-30T06:44:19.407ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
132392025-04-30T06:44:19.409ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "ffc60439-4e64-4ea0-a16b-6daba9929b7a", "content-length": "734", "date": "Wed, 30 Apr 2025 06:44:18 GMT"} })
132402025-04-30T06:44:19.409ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0/component/sp/caboose?firmware_slot=0
132412025-04-30T06:44:19.410ZDEBGtest_utilization (clickhouse-client): inserted rows into table collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 4d1485f0-110e-4161-9a33-c4610d68be5d n_rows = 1 table_name = measurements_i64
132422025-04-30T06:44:19.410ZTRACtest_utilization (results-sink): successfully inserted samples collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
132432025-04-30T06:44:19.411ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "272a2682-717f-4fec-9b2e-be4a7077c991", "content-length": "111", "date": "Wed, 30 Apr 2025 06:44:18 GMT"} })
132442025-04-30T06:44:19.412ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0/component/sp/caboose?firmware_slot=1
132452025-04-30T06:44:19.414ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "3b65cbbd-8af3-4d8b-8b6c-01f62288a7e1", "content-length": "111", "date": "Wed, 30 Apr 2025 06:44:18 GMT"} })
132462025-04-30T06:44:19.414ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0/component/rot/caboose?firmware_slot=0
132472025-04-30T06:44:19.416ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "79ef8fd5-ac43-4e8d-9585-633df9482944", "content-length": "170", "date": "Wed, 30 Apr 2025 06:44:18 GMT"} })
132482025-04-30T06:44:19.416ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0/component/rot/caboose?firmware_slot=1
132492025-04-30T06:44:19.422ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "de649360-470d-43e2-8dda-5fcc28c56110", "content-length": "170", "date": "Wed, 30 Apr 2025 06:44:18 GMT"} })
132502025-04-30T06:44:19.422ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0/component/stage0/caboose?firmware_slot=0
132512025-04-30T06:44:19.424ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "8921ca3a-ff4b-42fa-ba77-b4b1274a8495", "content-length": "179", "date": "Wed, 30 Apr 2025 06:44:18 GMT"} })
132522025-04-30T06:44:19.424ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0/component/stage0/caboose?firmware_slot=1
132532025-04-30T06:44:19.428ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "1c3e3171-05f1-4443-ae37-2436d1c4daef", "content-length": "179", "date": "Wed, 30 Apr 2025 06:44:18 GMT"} })
132542025-04-30T06:44:19.429ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0/component/rot/cmpa
132552025-04-30T06:44:19.431ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "73a89eff-425b-4ae2-8b83-ed9f1f5c8ea8", "content-length": "702", "date": "Wed, 30 Apr 2025 06:44:18 GMT"} })
132562025-04-30T06:44:19.431ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0/component/rot/cfpa
132572025-04-30T06:44:19.432ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "ebd73ae4-a951-45eb-b5a9-ad651990535d", "content-length": "727", "date": "Wed, 30 Apr 2025 06:44:18 GMT"} })
132582025-04-30T06:44:19.432ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0/component/rot/cfpa
132592025-04-30T06:44:19.433ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "8c790702-ed09-4679-aab7-754b07e9fb0e", "content-length": "729", "date": "Wed, 30 Apr 2025 06:44:18 GMT"} })
132602025-04-30T06:44:19.433ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0/component/rot/cfpa
132612025-04-30T06:44:19.435ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "ce923b10-744e-48ab-b98d-01aadca6cd86", "content-length": "728", "date": "Wed, 30 Apr 2025 06:44:18 GMT"} })
132622025-04-30T06:44:19.435ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1
132632025-04-30T06:44:19.459ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1", status: 200, headers: {"content-type": "application/json", "x-request-id": "65ce67f1-89e4-4f51-8d10-3aa226b938ac", "content-length": "734", "date": "Wed, 30 Apr 2025 06:44:18 GMT"} })
132642025-04-30T06:44:19.460ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1/component/sp/caboose?firmware_slot=0
132652025-04-30T06:44:19.463ZDEBGtest_utilization: attempt to fetch latest inventory collection result = Ok("found")
132662025-04-30T06:44:19.473ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "7f133d5d-3961-4638-bbda-6d4bb2f74e40", "content-length": "111", "date": "Wed, 30 Apr 2025 06:44:18 GMT"} })
132672025-04-30T06:44:19.475ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1/component/sp/caboose?firmware_slot=1
132682025-04-30T06:44:19.480ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "733fc34d-0005-4f4b-b06c-980f3b24556e", "content-length": "111", "date": "Wed, 30 Apr 2025 06:44:18 GMT"} })
132692025-04-30T06:44:19.480ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1/component/rot/caboose?firmware_slot=0
132702025-04-30T06:44:19.482ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "b36afeed-de79-458a-b0b6-ac2b47a9ebfd", "content-length": "170", "date": "Wed, 30 Apr 2025 06:44:18 GMT"} })
132712025-04-30T06:44:19.482ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1/component/rot/caboose?firmware_slot=1
132722025-04-30T06:44:19.484ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "db060045-9bd7-4da4-b6c5-703004305d76", "content-length": "170", "date": "Wed, 30 Apr 2025 06:44:18 GMT"} })
132732025-04-30T06:44:19.484ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1/component/stage0/caboose?firmware_slot=0
132742025-04-30T06:44:19.488ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "284545f5-9776-49db-bf06-ed614c9e291f", "content-length": "179", "date": "Wed, 30 Apr 2025 06:44:18 GMT"} })
132752025-04-30T06:44:19.488ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1/component/stage0/caboose?firmware_slot=1
132762025-04-30T06:44:19.491ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "0d6a6d5d-919c-4c60-be28-dab8d905d34b", "content-length": "179", "date": "Wed, 30 Apr 2025 06:44:18 GMT"} })
132772025-04-30T06:44:19.491ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1/component/rot/cmpa
132782025-04-30T06:44:19.493ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "6102a985-209d-4c44-ad7d-f0b9ce26dfb5", "content-length": "702", "date": "Wed, 30 Apr 2025 06:44:18 GMT"} })
132792025-04-30T06:44:19.493ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1/component/rot/cfpa
132802025-04-30T06:44:19.494ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "3352cd15-6e50-4736-b049-63b7086d8287", "content-length": "727", "date": "Wed, 30 Apr 2025 06:44:18 GMT"} })
132812025-04-30T06:44:19.494ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1/component/rot/cfpa
132822025-04-30T06:44:19.495ZWARNtest_utilization (results-sink-cluster): failed to insert some results into metric DB collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 error = Failed to check out connection to database: No backends found for this service file = oximeter/collector/src/results_sink.rs:92
132832025-04-30T06:44:19.496ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "f354e7f4-d7db-40a6-acef-aaaed0115202", "content-length": "729", "date": "Wed, 30 Apr 2025 06:44:18 GMT"} })
132842025-04-30T06:44:19.496ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1/component/rot/cfpa
132852025-04-30T06:44:19.497ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "9fbddb98-7e01-4f95-bec3-d9874e7e5823", "content-length": "728", "date": "Wed, 30 Apr 2025 06:44:18 GMT"} })
132862025-04-30T06:44:19.497ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0
132872025-04-30T06:44:19.499ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "9ea0abc2-1fbc-4f09-876f-4c718723c350", "content-length": "723", "date": "Wed, 30 Apr 2025 06:44:18 GMT"} })
132882025-04-30T06:44:19.499ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0/component/sp/caboose?firmware_slot=0
132892025-04-30T06:44:19.503ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "5db27fc4-fcd3-40dc-8fa4-334517eed811", "content-length": "109", "date": "Wed, 30 Apr 2025 06:44:18 GMT"} })
132902025-04-30T06:44:19.504ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0/component/sp/caboose?firmware_slot=1
132912025-04-30T06:44:19.507ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "2bd6af17-98a6-4c47-b36e-5e372fd01606", "content-length": "109", "date": "Wed, 30 Apr 2025 06:44:18 GMT"} })
132922025-04-30T06:44:19.507ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0/component/rot/caboose?firmware_slot=0
132932025-04-30T06:44:19.512ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "869286fe-b5c4-43d8-8395-afe718524e46", "content-length": "169", "date": "Wed, 30 Apr 2025 06:44:18 GMT"} })
132942025-04-30T06:44:19.512ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0/component/rot/caboose?firmware_slot=1
132952025-04-30T06:44:19.514ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "2b33db15-48f2-4dfd-a1e0-8386b25381b0", "content-length": "169", "date": "Wed, 30 Apr 2025 06:44:18 GMT"} })
132962025-04-30T06:44:19.515ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0/component/stage0/caboose?firmware_slot=0
132972025-04-30T06:44:19.515ZTRACtest_utilization (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory
132982025-04-30T06:44:19.515ZTRACtest_utilization (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
132992025-04-30T06:44:19.515ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
133002025-04-30T06:44:19.515ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
133012025-04-30T06:44:19.517ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
133022025-04-30T06:44:19.519ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "1266baf4-2da6-4a97-8d9e-a150290d9724", "content-length": "178", "date": "Wed, 30 Apr 2025 06:44:18 GMT"} })
133032025-04-30T06:44:19.520ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0/component/stage0/caboose?firmware_slot=1
133042025-04-30T06:44:19.520ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
133052025-04-30T06:44:19.531ZDEBGtest_utilization (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory result = Ok(())
133062025-04-30T06:44:19.531ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
133072025-04-30T06:44:19.531ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
133082025-04-30T06:44:19.533ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
133092025-04-30T06:44:19.536ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
133102025-04-30T06:44:19.536ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
133112025-04-30T06:44:19.536ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
133122025-04-30T06:44:19.539ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "6c769f72-9457-4a92-8a5b-7e9f9664420e", "content-length": "178", "date": "Wed, 30 Apr 2025 06:44:18 GMT"} })
133132025-04-30T06:44:19.539ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0/component/rot/cmpa
133142025-04-30T06:44:19.541ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "a22f6554-727d-43bf-a672-69a6d1bc6f45", "content-length": "702", "date": "Wed, 30 Apr 2025 06:44:18 GMT"} })
133152025-04-30T06:44:19.541ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0/component/rot/cfpa
133162025-04-30T06:44:19.542ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "a357c7b2-7538-42d2-be71-6bd39ca10d11", "content-length": "727", "date": "Wed, 30 Apr 2025 06:44:18 GMT"} })
133172025-04-30T06:44:19.543ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0/component/rot/cfpa
133182025-04-30T06:44:19.546ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "84f50a04-fc69-402e-9d15-f00fd0bcbfca", "content-length": "729", "date": "Wed, 30 Apr 2025 06:44:18 GMT"} })
133192025-04-30T06:44:19.546ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0/component/rot/cfpa
133202025-04-30T06:44:19.547ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "dba90c6e-119a-43bf-b38d-e90129b36b64", "content-length": "728", "date": "Wed, 30 Apr 2025 06:44:18 GMT"} })
133212025-04-30T06:44:19.547ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1
133222025-04-30T06:44:19.549ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1", status: 200, headers: {"content-type": "application/json", "x-request-id": "607ff2e9-fc75-4a57-a289-ec71755220c1", "content-length": "723", "date": "Wed, 30 Apr 2025 06:44:18 GMT"} })
133232025-04-30T06:44:19.551ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1/component/sp/caboose?firmware_slot=0
133242025-04-30T06:44:19.555ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "30c279d5-7d57-4ca0-bd4b-72e01d03ef91", "content-length": "109", "date": "Wed, 30 Apr 2025 06:44:18 GMT"} })
133252025-04-30T06:44:19.555ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1/component/sp/caboose?firmware_slot=1
133262025-04-30T06:44:19.590ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "1e183f9e-a039-4b41-b8c5-9d6d4757200f", "content-length": "109", "date": "Wed, 30 Apr 2025 06:44:18 GMT"} })
133272025-04-30T06:44:19.590ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1/component/rot/caboose?firmware_slot=0
133282025-04-30T06:44:19.609ZDEBGtest_utilization: attempt to fetch latest inventory collection result = Ok("found")
133292025-04-30T06:44:19.653ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "fc48c30a-a3dd-4f4a-9517-c8b2b50d4a9a", "content-length": "169", "date": "Wed, 30 Apr 2025 06:44:18 GMT"} })
133302025-04-30T06:44:19.653ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1/component/rot/caboose?firmware_slot=1
133312025-04-30T06:44:19.660ZTRACtest_utilization (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory
133322025-04-30T06:44:19.660ZTRACtest_utilization (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
133332025-04-30T06:44:19.660ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
133342025-04-30T06:44:19.660ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
133352025-04-30T06:44:19.664ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
133362025-04-30T06:44:19.667ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
133372025-04-30T06:44:19.674ZDEBGtest_utilization (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory result = Ok(())
133382025-04-30T06:44:19.674ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
133392025-04-30T06:44:19.674ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
133402025-04-30T06:44:19.675ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
133412025-04-30T06:44:19.678ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "63327764-00a0-4adf-b13e-11ab6a4e8ea8", "content-length": "169", "date": "Wed, 30 Apr 2025 06:44:18 GMT"} })
133422025-04-30T06:44:19.678ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1/component/stage0/caboose?firmware_slot=0
133432025-04-30T06:44:19.678ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
133442025-04-30T06:44:19.678ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
133452025-04-30T06:44:19.679ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
133462025-04-30T06:44:19.683ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "005b9674-7949-49d6-abc3-6e7e16762b1a", "content-length": "178", "date": "Wed, 30 Apr 2025 06:44:18 GMT"} })
133472025-04-30T06:44:19.683ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1/component/stage0/caboose?firmware_slot=1
133482025-04-30T06:44:19.688ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "9cc7c870-3be2-4e85-85e3-f00bc4149794", "content-length": "178", "date": "Wed, 30 Apr 2025 06:44:18 GMT"} })
133492025-04-30T06:44:19.688ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1/component/rot/cmpa
133502025-04-30T06:44:19.689ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "81ef9c06-3061-4d6a-a085-bd0141e9eaf0", "content-length": "702", "date": "Wed, 30 Apr 2025 06:44:18 GMT"} })
133512025-04-30T06:44:19.689ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1/component/rot/cfpa
133522025-04-30T06:44:19.691ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "b3fbc028-0c6d-4577-8165-ee3f8c3d068a", "content-length": "727", "date": "Wed, 30 Apr 2025 06:44:18 GMT"} })
133532025-04-30T06:44:19.692ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1/component/rot/cfpa
133542025-04-30T06:44:19.693ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "9bed33e2-d2c5-45dd-9373-8f7705b48b2d", "content-length": "729", "date": "Wed, 30 Apr 2025 06:44:18 GMT"} })
133552025-04-30T06:44:19.693ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1/component/rot/cfpa
133562025-04-30T06:44:19.694ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "800511f4-66bb-4e82-832b-9b689f2e1d4b", "content-length": "728", "date": "Wed, 30 Apr 2025 06:44:18 GMT"} })
133572025-04-30T06:44:19.694ZTRACtest_utilization (BackgroundTasks): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet
133582025-04-30T06:44:19.694ZTRACtest_utilization (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
133592025-04-30T06:44:19.694ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
133602025-04-30T06:44:19.695ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
133612025-04-30T06:44:19.695ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
133622025-04-30T06:44:19.698ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
133632025-04-30T06:44:19.701ZDEBGtest_utilization (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet result = Ok(())
133642025-04-30T06:44:19.701ZTRACtest_utilization (BackgroundTasks): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet
133652025-04-30T06:44:19.701ZTRACtest_utilization (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
133662025-04-30T06:44:19.701ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
133672025-04-30T06:44:19.701ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
133682025-04-30T06:44:19.701ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
133692025-04-30T06:44:19.708ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
133702025-04-30T06:44:19.716ZDEBGtest_utilization (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet result = Ok(())
133712025-04-30T06:44:19.717ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
133722025-04-30T06:44:19.717ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
133732025-04-30T06:44:19.717ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
133742025-04-30T06:44:19.815ZDEBGtest_utilization (BackgroundTasks): begin collection from Sled Agent background_task = inventory_collection sled_agent_url = http://[::1]:39365
133752025-04-30T06:44:19.815ZDEBGtest_utilization (BackgroundTasks): client request SledAgent = http://[::1]:39365 background_task = inventory_collection body = None method = GET uri = http://[::1]:39365/inventory
133762025-04-30T06:44:19.816ZINFOtest_utilization (dropshot): accepted connection local_addr = [::1]:39365 remote_addr = [::1]:37396 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
133772025-04-30T06:44:19.817ZTRACtest_utilization (dropshot): incoming request local_addr = [::1]:39365 method = GET remote_addr = [::1]:37396 req_id = bd03f645-8474-421b-a071-470aabe48747 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /inventory
133782025-04-30T06:44:19.817ZINFOtest_utilization (dropshot): request completed latency_us = 514 local_addr = [::1]:39365 method = GET remote_addr = [::1]:37396 req_id = bd03f645-8474-421b-a071-470aabe48747 response_code = 200 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /inventory
133792025-04-30T06:44:19.817ZDEBGtest_utilization (BackgroundTasks): client response SledAgent = http://[::1]:39365 background_task = inventory_collection result = Ok(Response { url: "http://[::1]:39365/inventory", status: 200, headers: {"content-type": "application/json", "x-request-id": "bd03f645-8474-421b-a071-470aabe48747", "content-length": "3141", "date": "Wed, 30 Apr 2025 06:44:18 GMT"} })
133802025-04-30T06:44:19.819ZDEBGtest_utilization (BackgroundTasks): begin collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
133812025-04-30T06:44:19.819ZDEBGtest_utilization (BackgroundTasks): end collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
133822025-04-30T06:44:19.819ZDEBGtest_utilization (BackgroundTasks): finished collection background_task = inventory_collection
133832025-04-30T06:44:19.825ZTRACtest_utilization (BackgroundTasks): authorize begin action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory
133842025-04-30T06:44:19.826ZTRACtest_utilization (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
133852025-04-30T06:44:19.830ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
133862025-04-30T06:44:19.830ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
133872025-04-30T06:44:19.832ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
133882025-04-30T06:44:19.840ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
133892025-04-30T06:44:19.850ZDEBGtest_utilization (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory result = Ok(())
133902025-04-30T06:44:19.850ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
133912025-04-30T06:44:19.850ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
133922025-04-30T06:44:19.852ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
133932025-04-30T06:44:19.854ZDEBGtest_utilization: attempt to fetch latest inventory collection result = Ok("found")
133942025-04-30T06:44:19.890ZWARNtest_utilization (datastore): Non-retryable transaction failure err = DatabaseError(UniqueViolation, "duplicate key value violates unique constraint \\"inv_caboose_pkey\\"") transaction = inventory_insert_collection
133952025-04-30T06:44:19.890ZWARNtest_utilization (BackgroundTasks): inventory collection failed background_task = inventory_collection error = failed to collect inventory: saving inventory to database: Internal Error: unexpected database error: duplicate key value violates unique constraint "inv_caboose_pkey"
133962025-04-30T06:44:19.890ZDEBGtest_utilization (BackgroundTasks): activation complete background_task = inventory_collection elapsed = 605.842502ms iteration = 35 status = {"error":"failed to collect inventory: saving inventory to database: Internal Error: unexpected database error: duplicate key value violates unique constraint \\"inv_caboose_pkey\\""}
133972025-04-30T06:44:19.891ZDEBGtest_utilization (BackgroundTasks): activating background_task = inventory_collection iteration = 36 reason = Signaled
133982025-04-30T06:44:19.891ZTRACtest_utilization (BackgroundTasks): authorize begin action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory
133992025-04-30T06:44:19.891ZTRACtest_utilization (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
134002025-04-30T06:44:19.891ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
134012025-04-30T06:44:19.891ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
134022025-04-30T06:44:19.895ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
134032025-04-30T06:44:19.915ZTRACtest_utilization (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory
134042025-04-30T06:44:19.915ZTRACtest_utilization (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
134052025-04-30T06:44:19.915ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
134062025-04-30T06:44:19.915ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
134072025-04-30T06:44:19.916ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
134082025-04-30T06:44:19.916ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
134092025-04-30T06:44:19.920ZDEBGtest_utilization (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory result = Ok(())
134102025-04-30T06:44:19.920ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
134112025-04-30T06:44:19.920ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
134122025-04-30T06:44:19.923ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
134132025-04-30T06:44:19.926ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
134142025-04-30T06:44:19.933ZDEBGtest_utilization (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory result = Ok(())
134152025-04-30T06:44:19.934ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
134162025-04-30T06:44:19.934ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
134172025-04-30T06:44:19.935ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
134182025-04-30T06:44:19.937ZDEBGtest_utilization (BackgroundTasks): inventory_prune_one: nothing eligible for removal (too few) background_task = inventory_collection candidates = [(13d8b8d9-6c6b-47a6-9190-952544edb75b, 0)]
134192025-04-30T06:44:19.937ZTRACtest_utilization (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal
134202025-04-30T06:44:19.938ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
134212025-04-30T06:44:19.938ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
134222025-04-30T06:44:19.939ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
134232025-04-30T06:44:19.940ZTRACtest_utilization (dns): buffer buffer = Length: 67 (0x43) bytes\n0000: 2d 3b 01 00 00 01 00 00 00 00 00 01 04 5f 6d 67 -;..........._mg\n0010: 73 04 5f 74 63 70 0d 63 6f 6e 74 72 6f 6c 2d 70 s._tcp.control-p\n0020: 6c 61 6e 65 05 6f 78 69 64 65 08 69 6e 74 65 72 lane.oxide.inter\n0030: 6e 61 6c 00 00 21 00 01 00 00 29 04 d0 00 00 00 nal..!....).....\n0040: 00 00 00 ... kind = dns peer_addr = [::1]:65494 req_id = 68e3b179-aefc-4b2e-ad46-97b894063daa
134242025-04-30T06:44:19.940ZDEBGtest_utilization (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 11579,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_mgs._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_mgs._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 4,\n 95,\n 109,\n 103,\n 115,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:65494 req_id = 68e3b179-aefc-4b2e-ad46-97b894063daa
134252025-04-30T06:44:19.940ZDEBGtest_utilization (store): query key key = _mgs._tcp
134262025-04-30T06:44:19.941ZDEBGtest_utilization (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
134272025-04-30T06:44:19.941ZDEBGtest_utilization (dns): dns response additional_records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal"), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] kind = dns peer_addr = [::1]:65494 query = LowerQuery { name: LowerName(Name("_mgs._tcp.control-plane.oxide.internal.")), original: Query { name: Name("_mgs._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN } } records = [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 48759, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal") })) }] req_id = 68e3b179-aefc-4b2e-ad46-97b894063daa
134282025-04-30T06:44:19.942ZDEBGtest_utilization (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgs._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 48759, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }], valid_until: Instant { tv_sec: 4324, tv_nsec: 741281090 } })
134292025-04-30T06:44:19.942ZTRACtest_utilization (DnsResolver): lookup_service_targets: looking up SRV target
134302025-04-30T06:44:19.942ZTRACtest_utilization (dns): buffer buffer = Length: 108 (0x6c) bytes\n0000: 5e 69 01 00 00 01 00 00 00 00 00 01 2d 64 65 6e ^i..........-den\n0010: 64 72 69 74 65 2d 62 36 64 36 35 33 34 31 2d 31 drite-b6d65341-1\n0020: 36 37 63 2d 34 31 64 66 2d 39 62 35 63 2d 34 31 67c-41df-9b5c-41\n0030: 63 64 65 64 39 39 63 32 32 39 04 68 6f 73 74 0d cded99c229.host.\n0040: 63 6f 6e 74 72 6f 6c 2d 70 6c 61 6e 65 05 6f 78 control-plane.ox\n0050: 69 64 65 08 69 6e 74 65 72 6e 61 6c 00 00 1c 00 ide.internal....\n0060: 01 00 00 29 04 d0 00 00 00 00 00 00 ...)........ kind = dns peer_addr = [::1]:61197 req_id = ed439af5-42c2-45c3-9d72-4a451cd92ec7
134312025-04-30T06:44:19.943ZDEBGtest_utilization (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 24169,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n query_type: AAAA,\n query_class: IN,\n },\n },\n original: [\n 45,\n 100,\n 101,\n 110,\n 100,\n 114,\n 105,\n 116,\n 101,\n 45,\n 98,\n 54,\n 100,\n 54,\n 53,\n 51,\n 52,\n 49,\n 45,\n 49,\n 54,\n 55,\n 99,\n 45,\n 52,\n 49,\n 100,\n 102,\n 45,\n 57,\n 98,\n 53,\n 99,\n 45,\n 52,\n 49,\n 99,\n 100,\n 101,\n 100,\n 57,\n 57,\n 99,\n 50,\n 50,\n 57,\n 4,\n 104,\n 111,\n 115,\n 116,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 28,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:61197 req_id = ed439af5-42c2-45c3-9d72-4a451cd92ec7
134322025-04-30T06:44:19.944ZDEBGtest_utilization (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
134332025-04-30T06:44:19.944ZDEBGtest_utilization (dns): dns response additional_records = [] kind = dns peer_addr = [::1]:61197 query = LowerQuery { name: LowerName(Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.")), original: Query { name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), query_type: AAAA, query_class: IN } } records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] req_id = ed439af5-42c2-45c3-9d72-4a451cd92ec7
134342025-04-30T06:44:20.003ZTRACtest_utilization (DnsResolver): lookup_all_socket_v6 srv dns_name = _clickhouse-admin-keeper._tcp.control-plane.oxide.internal
134352025-04-30T06:44:20.004ZINFOtest_utilization (oximeter-agent): refreshing list of producers from Nexus collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 file = oximeter/collector/src/agent.rs:562
134362025-04-30T06:44:20.005ZDEBGtest_utilization (collection-task): sent timer-based collection request to the collection task collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
134372025-04-30T06:44:20.006ZDEBGtest_utilization (collection-task): collecting from producer collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
134382025-04-30T06:44:20.007ZDEBGtest_utilization (oximeter): client request body = None method = GET uri = http://127.0.0.1:32956/metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?sort_by=id_ascending
134392025-04-30T06:44:20.007ZTRACtest_utilization (dropshot_internal): incoming request local_addr = 127.0.0.1:32956 method = GET remote_addr = 127.0.0.1:55797 req_id = bb0e6bc0-5307-4002-b65d-6075f8bee9ba uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?sort_by=id_ascending
134402025-04-30T06:44:20.008ZTRACtest_utilization (dns): buffer buffer = Length: 87 (0x57) bytes\n0000: de 08 01 00 00 01 00 00 00 00 00 01 18 5f 63 6c ............._cl\n0010: 69 63 6b 68 6f 75 73 65 2d 61 64 6d 69 6e 2d 6b ickhouse-admin-k\n0020: 65 65 70 65 72 04 5f 74 63 70 0d 63 6f 6e 74 72 eeper._tcp.contr\n0030: 6f 6c 2d 70 6c 61 6e 65 05 6f 78 69 64 65 08 69 ol-plane.oxide.i\n0040: 6e 74 65 72 6e 61 6c 00 00 21 00 01 00 00 29 04 nternal..!....).\n0050: d0 00 00 00 00 00 00 ....... kind = dns peer_addr = [::1]:49813 req_id = 4bb39f72-01d5-4528-b1ae-d788bd481f93
134412025-04-30T06:44:20.009ZDEBGtest_utilization (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 56840,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 24,\n 95,\n 99,\n 108,\n 105,\n 99,\n 107,\n 104,\n 111,\n 117,\n 115,\n 101,\n 45,\n 97,\n 100,\n 109,\n 105,\n 110,\n 45,\n 107,\n 101,\n 101,\n 112,\n 101,\n 114,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:49813 req_id = 4bb39f72-01d5-4528-b1ae-d788bd481f93
134422025-04-30T06:44:20.009ZDEBGtest_utilization (store): query key key = _clickhouse-admin-keeper._tcp
134432025-04-30T06:44:20.010ZERROtest_utilization (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 56840, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 1, }, query: WireQuery { query: LowerQuery { name: LowerName( Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), ), original: Query { name: Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN, }, }, original: [ 24, 95, 99, 108, 105, 99, 107, 104, 111, 117, 115, 101, 45, 97, 100, 109, 105, 110, 45, 107, 101, 101, 112, 101, 114, 4, 95, 116, 99, 112, 13, 99, 111, 110, 116, 114, 111, 108, 45, 112, 108, 97, 110, 101, 5, 111, 120, 105, 100, 101, 8, 105, 110, 116, 101, 114, 110, 97, 108, 0, 0, 33, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: Some( Edns { rcode_high: 0, version: 0, dnssec_ok: false, max_payload: 1232, options: OPT { options: {}, }, }, ), } NXDOMAIN: no records found for name: "_clickhouse-admin-keeper._tcp.control-plane.oxide.internal." kind = dns peer_addr = [::1]:49813 req_id = 4bb39f72-01d5-4528-b1ae-d788bd481f93
134442025-04-30T06:44:20.065ZTRACtest_utilization (dropshot_internal): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = 127.0.0.1:32956 method = GET remote_addr = 127.0.0.1:55797 req_id = bb0e6bc0-5307-4002-b65d-6075f8bee9ba resource = Database uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?sort_by=id_ascending
134452025-04-30T06:44:20.065ZDEBGtest_utilization (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = 127.0.0.1:32956 method = GET remote_addr = 127.0.0.1:55797 req_id = bb0e6bc0-5307-4002-b65d-6075f8bee9ba roles = RoleSet { roles: {} } uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?sort_by=id_ascending
134462025-04-30T06:44:20.066ZDEBGtest_utilization (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = 127.0.0.1:32956 method = GET remote_addr = 127.0.0.1:55797 req_id = bb0e6bc0-5307-4002-b65d-6075f8bee9ba resource = Database result = Ok(()) uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?sort_by=id_ascending
134472025-04-30T06:44:20.068ZDEBGtest_utilization (BackgroundTasks): begin collection background_task = inventory_collection
134482025-04-30T06:44:20.068ZDEBGtest_utilization (BackgroundTasks): begin collection from MGS background_task = inventory_collection mgs_url = http://[::1]:48759
134492025-04-30T06:44:20.068ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/ignition
134502025-04-30T06:44:20.068ZINFOtest_utilization (dropshot): accepted connection kind = pantry local_addr = [::1]:45558 remote_addr = [::1]:59566 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
134512025-04-30T06:44:20.069ZTRACtest_utilization (collection-task): sent collection request to producer collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
134522025-04-30T06:44:20.070ZDEBGtest_utilization (collection-task): collected results from producer collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 n_results = 1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
134532025-04-30T06:44:20.070ZTRACtest_utilization (collection-task): forwarded results to main collection loop collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
134542025-04-30T06:44:20.070ZTRACtest_utilization (collection-task): top of inner collection loop, waiting for next request collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
134552025-04-30T06:44:20.071ZTRACtest_utilization (dropshot): incoming request kind = pantry local_addr = [::1]:45558 method = GET remote_addr = [::1]:59566 req_id = abc1e6c7-b43f-4382-a645-4744109071f0 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /crucible/pantry/0
134562025-04-30T06:44:20.071ZINFOtest_utilization (dropshot): request completed kind = pantry latency_us = 388 local_addr = [::1]:45558 method = GET remote_addr = [::1]:59566 req_id = abc1e6c7-b43f-4382-a645-4744109071f0 response_code = 200 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /crucible/pantry/0
134572025-04-30T06:44:20.073ZINFOtest_utilization (dropshot_internal): request completed latency_us = 65207 local_addr = 127.0.0.1:32956 method = GET remote_addr = 127.0.0.1:55797 req_id = bb0e6bc0-5307-4002-b65d-6075f8bee9ba response_code = 200 uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?sort_by=id_ascending
134582025-04-30T06:44:20.073ZDEBGtest_utilization (oximeter): client response result = Ok(Response { url: "http://127.0.0.1:32956/metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?sort_by=id_ascending", status: 200, headers: {"content-type": "application/json", "x-request-id": "bb0e6bc0-5307-4002-b65d-6075f8bee9ba", "content-length": "405", "date": "Wed, 30 Apr 2025 06:44:20 GMT"} })
134592025-04-30T06:44:20.074ZDEBGtest_utilization (oximeter): client request body = None method = GET uri = http://127.0.0.1:32956/metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJhYzU1ZDhkOS1hMjk4LTQ1MzQtODk4Yi1iZWVlMWM5ZTRlNTIifX0%3D
134602025-04-30T06:44:20.075ZTRACtest_utilization (dropshot_internal): incoming request local_addr = 127.0.0.1:32956 method = GET remote_addr = 127.0.0.1:55797 req_id = 8d47bace-a38b-43e0-be13-bd5bd2f8ac34 uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJhYzU1ZDhkOS1hMjk4LTQ1MzQtODk4Yi1iZWVlMWM5ZTRlNTIifX0%3D
134612025-04-30T06:44:20.075ZTRACtest_utilization (dropshot_internal): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = 127.0.0.1:32956 method = GET remote_addr = 127.0.0.1:55797 req_id = 8d47bace-a38b-43e0-be13-bd5bd2f8ac34 resource = Database uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJhYzU1ZDhkOS1hMjk4LTQ1MzQtODk4Yi1iZWVlMWM5ZTRlNTIifX0%3D
134622025-04-30T06:44:20.075ZDEBGtest_utilization (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = 127.0.0.1:32956 method = GET remote_addr = 127.0.0.1:55797 req_id = 8d47bace-a38b-43e0-be13-bd5bd2f8ac34 roles = RoleSet { roles: {} } uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJhYzU1ZDhkOS1hMjk4LTQ1MzQtODk4Yi1iZWVlMWM5ZTRlNTIifX0%3D
134632025-04-30T06:44:20.084ZDEBGtest_utilization (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = 127.0.0.1:32956 method = GET remote_addr = 127.0.0.1:55797 req_id = 8d47bace-a38b-43e0-be13-bd5bd2f8ac34 resource = Database result = Ok(()) uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJhYzU1ZDhkOS1hMjk4LTQ1MzQtODk4Yi1iZWVlMWM5ZTRlNTIifX0%3D
134642025-04-30T06:44:20.084ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/ignition", status: 200, headers: {"content-type": "application/json", "x-request-id": "5e5451c7-c976-414c-8e78-2913162c7db2", "content-length": "835", "date": "Wed, 30 Apr 2025 06:44:20 GMT"} })
134652025-04-30T06:44:20.084ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0
134662025-04-30T06:44:20.084ZINFOtest_utilization (dropshot_internal): request completed latency_us = 9286 local_addr = 127.0.0.1:32956 method = GET remote_addr = 127.0.0.1:55797 req_id = 8d47bace-a38b-43e0-be13-bd5bd2f8ac34 response_code = 200 uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJhYzU1ZDhkOS1hMjk4LTQ1MzQtODk4Yi1iZWVlMWM5ZTRlNTIifX0%3D
134672025-04-30T06:44:20.085ZDEBGtest_utilization (oximeter): client response result = Ok(Response { url: "http://127.0.0.1:32956/metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJhYzU1ZDhkOS1hMjk4LTQ1MzQtODk4Yi1iZWVlMWM5ZTRlNTIifX0%3D", status: 200, headers: {"content-type": "application/json", "x-request-id": "8d47bace-a38b-43e0-be13-bd5bd2f8ac34", "content-length": "29", "date": "Wed, 30 Apr 2025 06:44:20 GMT"} })
134682025-04-30T06:44:20.086ZTRACtest_utilization (oximeter-agent): ignoring request to update existing metric producer, since the endpoint information is the same as the existing collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
134692025-04-30T06:44:20.086ZTRACtest_utilization (oximeter-agent): ignoring request to update existing metric producer, since the endpoint information is the same as the existing collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = ac55d8d9-a298-4534-898b-beee1c9e4e52
134702025-04-30T06:44:20.086ZINFOtest_utilization (oximeter-agent): refreshed list of producers from Nexus collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 file = oximeter/collector/src/agent.rs:637 n_current_tasks = 2 n_pruned_tasks = 0
134712025-04-30T06:44:20.087ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "e3dbd748-9d5c-4886-ad7c-d14c969c10db", "content-length": "734", "date": "Wed, 30 Apr 2025 06:44:20 GMT"} })
134722025-04-30T06:44:20.087ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0/component/sp/caboose?firmware_slot=0
134732025-04-30T06:44:20.093ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "96b8f831-5000-40a5-bc7e-8d10be16a0f4", "content-length": "111", "date": "Wed, 30 Apr 2025 06:44:20 GMT"} })
134742025-04-30T06:44:20.093ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0/component/sp/caboose?firmware_slot=1
134752025-04-30T06:44:20.097ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "c693d117-37ce-4a64-ac2e-2e112f2d7a1a", "content-length": "111", "date": "Wed, 30 Apr 2025 06:44:20 GMT"} })
134762025-04-30T06:44:20.097ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0/component/rot/caboose?firmware_slot=0
134772025-04-30T06:44:20.103ZDEBGtest_utilization: attempt to fetch latest inventory collection result = Ok("found")
134782025-04-30T06:44:20.103ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "5d6e6fc8-9d49-4498-aa9c-e367a4e1e7e7", "content-length": "170", "date": "Wed, 30 Apr 2025 06:44:20 GMT"} })
134792025-04-30T06:44:20.103ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0/component/rot/caboose?firmware_slot=1
134802025-04-30T06:44:20.106ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "fe22762d-4c34-4c05-8b98-9a8d55124f85", "content-length": "170", "date": "Wed, 30 Apr 2025 06:44:20 GMT"} })
134812025-04-30T06:44:20.107ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0/component/stage0/caboose?firmware_slot=0
134822025-04-30T06:44:20.109ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "b02b3077-a186-4b34-89ed-a8a2cedd2bbe", "content-length": "179", "date": "Wed, 30 Apr 2025 06:44:20 GMT"} })
134832025-04-30T06:44:20.109ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0/component/stage0/caboose?firmware_slot=1
134842025-04-30T06:44:20.111ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "a7385b03-e1ce-4044-8f6b-5a2261671452", "content-length": "179", "date": "Wed, 30 Apr 2025 06:44:20 GMT"} })
134852025-04-30T06:44:20.112ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0/component/rot/cmpa
134862025-04-30T06:44:20.113ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "7c3492b3-dc2b-4819-a821-34709055abd7", "content-length": "702", "date": "Wed, 30 Apr 2025 06:44:20 GMT"} })
134872025-04-30T06:44:20.113ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0/component/rot/cfpa
134882025-04-30T06:44:20.115ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "0e62b87c-438c-45bf-9877-71ae56d67b60", "content-length": "727", "date": "Wed, 30 Apr 2025 06:44:20 GMT"} })
134892025-04-30T06:44:20.115ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0/component/rot/cfpa
134902025-04-30T06:44:20.120ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "4d1549b2-a8ae-4454-9d03-bc5c0d8e882a", "content-length": "729", "date": "Wed, 30 Apr 2025 06:44:20 GMT"} })
134912025-04-30T06:44:20.120ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0/component/rot/cfpa
134922025-04-30T06:44:20.121ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "cc680863-51a6-45f2-a210-de0b85d0537a", "content-length": "728", "date": "Wed, 30 Apr 2025 06:44:20 GMT"} })
134932025-04-30T06:44:20.121ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1
134942025-04-30T06:44:20.126ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1", status: 200, headers: {"content-type": "application/json", "x-request-id": "5fa9b954-e31c-4af2-9005-b17e90dc705e", "content-length": "734", "date": "Wed, 30 Apr 2025 06:44:20 GMT"} })
134952025-04-30T06:44:20.127ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1/component/sp/caboose?firmware_slot=0
134962025-04-30T06:44:20.131ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "07078e2c-1387-4f77-8371-3aaedd6faf5f", "content-length": "111", "date": "Wed, 30 Apr 2025 06:44:20 GMT"} })
134972025-04-30T06:44:20.132ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1/component/sp/caboose?firmware_slot=1
134982025-04-30T06:44:20.134ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "7e7a7ea7-f5a8-4cbc-897c-b9a93e12468f", "content-length": "111", "date": "Wed, 30 Apr 2025 06:44:20 GMT"} })
134992025-04-30T06:44:20.136ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1/component/rot/caboose?firmware_slot=0
135002025-04-30T06:44:20.139ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "77570222-8384-4a7b-a868-d09d06dc8dbb", "content-length": "170", "date": "Wed, 30 Apr 2025 06:44:20 GMT"} })
135012025-04-30T06:44:20.139ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1/component/rot/caboose?firmware_slot=1
135022025-04-30T06:44:20.144ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "607df57e-b0a2-463f-9cb2-4cf5fb487b11", "content-length": "170", "date": "Wed, 30 Apr 2025 06:44:20 GMT"} })
135032025-04-30T06:44:20.144ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1/component/stage0/caboose?firmware_slot=0
135042025-04-30T06:44:20.146ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "e17aebd7-399a-4e10-b594-2800afab9727", "content-length": "179", "date": "Wed, 30 Apr 2025 06:44:20 GMT"} })
135052025-04-30T06:44:20.146ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1/component/stage0/caboose?firmware_slot=1
135062025-04-30T06:44:20.148ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "eab3b2fb-0a32-4e0e-8eb5-3353735d5877", "content-length": "179", "date": "Wed, 30 Apr 2025 06:44:20 GMT"} })
135072025-04-30T06:44:20.148ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1/component/rot/cmpa
135082025-04-30T06:44:20.149ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "2742e41e-88ec-4f32-a566-91de4e135723", "content-length": "702", "date": "Wed, 30 Apr 2025 06:44:20 GMT"} })
135092025-04-30T06:44:20.150ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1/component/rot/cfpa
135102025-04-30T06:44:20.151ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "a64d7663-6645-4cdc-a791-ccee0f4a3a6c", "content-length": "727", "date": "Wed, 30 Apr 2025 06:44:20 GMT"} })
135112025-04-30T06:44:20.152ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1/component/rot/cfpa
135122025-04-30T06:44:20.153ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "70d3aaf2-8bee-4a3c-9dd8-dc6383975dd9", "content-length": "729", "date": "Wed, 30 Apr 2025 06:44:20 GMT"} })
135132025-04-30T06:44:20.153ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1/component/rot/cfpa
135142025-04-30T06:44:20.153ZTRACtest_utilization (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory
135152025-04-30T06:44:20.153ZTRACtest_utilization (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
135162025-04-30T06:44:20.154ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
135172025-04-30T06:44:20.154ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
135182025-04-30T06:44:20.154ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
135192025-04-30T06:44:20.156ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "1eb5a468-a768-45f9-b72c-475deee015ce", "content-length": "728", "date": "Wed, 30 Apr 2025 06:44:20 GMT"} })
135202025-04-30T06:44:20.158ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0
135212025-04-30T06:44:20.159ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
135222025-04-30T06:44:20.167ZDEBGtest_utilization (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory result = Ok(())
135232025-04-30T06:44:20.167ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
135242025-04-30T06:44:20.167ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
135252025-04-30T06:44:20.168ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
135262025-04-30T06:44:20.171ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
135272025-04-30T06:44:20.174ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
135282025-04-30T06:44:20.174ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
135292025-04-30T06:44:20.174ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "768b5cbf-d3f4-4c7e-a7a6-f5d7d62fed72", "content-length": "723", "date": "Wed, 30 Apr 2025 06:44:20 GMT"} })
135302025-04-30T06:44:20.174ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0/component/sp/caboose?firmware_slot=0
135312025-04-30T06:44:20.177ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "2381bb18-281e-41c0-84b8-95a0865b480e", "content-length": "109", "date": "Wed, 30 Apr 2025 06:44:20 GMT"} })
135322025-04-30T06:44:20.178ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0/component/sp/caboose?firmware_slot=1
135332025-04-30T06:44:20.183ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "068db830-b415-4a10-bd71-76eb36d926db", "content-length": "109", "date": "Wed, 30 Apr 2025 06:44:20 GMT"} })
135342025-04-30T06:44:20.183ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0/component/rot/caboose?firmware_slot=0
135352025-04-30T06:44:20.186ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "ee421bbd-1ffa-499e-b0bc-2cb4db515920", "content-length": "169", "date": "Wed, 30 Apr 2025 06:44:20 GMT"} })
135362025-04-30T06:44:20.186ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0/component/rot/caboose?firmware_slot=1
135372025-04-30T06:44:20.196ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "0f7d5f2a-95d3-4cb7-b2b7-9d6b819bdad3", "content-length": "169", "date": "Wed, 30 Apr 2025 06:44:20 GMT"} })
135382025-04-30T06:44:20.197ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0/component/stage0/caboose?firmware_slot=0
135392025-04-30T06:44:20.205ZDEBGtest_utilization: attempt to fetch latest inventory collection result = Ok("found")
135402025-04-30T06:44:20.206ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "01de7720-8c85-48b4-a8f5-96b607612b01", "content-length": "178", "date": "Wed, 30 Apr 2025 06:44:20 GMT"} })
135412025-04-30T06:44:20.206ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0/component/stage0/caboose?firmware_slot=1
135422025-04-30T06:44:20.209ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "e13d67b3-d71c-4d67-809d-c3f1e52c8fc0", "content-length": "178", "date": "Wed, 30 Apr 2025 06:44:20 GMT"} })
135432025-04-30T06:44:20.209ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0/component/rot/cmpa
135442025-04-30T06:44:20.211ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "dd2b97e5-a33b-48eb-9859-ee9bab528381", "content-length": "702", "date": "Wed, 30 Apr 2025 06:44:20 GMT"} })
135452025-04-30T06:44:20.212ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0/component/rot/cfpa
135462025-04-30T06:44:20.214ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "28bf47e1-5f5a-46da-b083-bdf2a0175f25", "content-length": "727", "date": "Wed, 30 Apr 2025 06:44:20 GMT"} })
135472025-04-30T06:44:20.216ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0/component/rot/cfpa
135482025-04-30T06:44:20.218ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "77637f97-fc12-43b8-ab3a-b6ddad524b7e", "content-length": "729", "date": "Wed, 30 Apr 2025 06:44:20 GMT"} })
135492025-04-30T06:44:20.219ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0/component/rot/cfpa
135502025-04-30T06:44:20.219ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "446f65a1-a93f-4997-b66b-9dbb593c862f", "content-length": "728", "date": "Wed, 30 Apr 2025 06:44:20 GMT"} })
135512025-04-30T06:44:20.220ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1
135522025-04-30T06:44:20.221ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1", status: 200, headers: {"content-type": "application/json", "x-request-id": "eb1e2c15-cf37-478d-b225-c2a31922762a", "content-length": "723", "date": "Wed, 30 Apr 2025 06:44:20 GMT"} })
135532025-04-30T06:44:20.221ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1/component/sp/caboose?firmware_slot=0
135542025-04-30T06:44:20.223ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "6ac90b0b-4ea6-45f3-8866-e45c8b763eb2", "content-length": "109", "date": "Wed, 30 Apr 2025 06:44:20 GMT"} })
135552025-04-30T06:44:20.224ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1/component/sp/caboose?firmware_slot=1
135562025-04-30T06:44:20.229ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "038b49ab-5cfc-4367-a811-c59fd86e795e", "content-length": "109", "date": "Wed, 30 Apr 2025 06:44:20 GMT"} })
135572025-04-30T06:44:20.229ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1/component/rot/caboose?firmware_slot=0
135582025-04-30T06:44:20.229ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "8b0569bc-80d0-4c5a-8948-63af0d698f99", "content-length": "169", "date": "Wed, 30 Apr 2025 06:44:20 GMT"} })
135592025-04-30T06:44:20.229ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1/component/rot/caboose?firmware_slot=1
135602025-04-30T06:44:20.235ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "bfd4bdbc-610e-4aa3-8a0f-02525e87e7d8", "content-length": "169", "date": "Wed, 30 Apr 2025 06:44:20 GMT"} })
135612025-04-30T06:44:20.235ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1/component/stage0/caboose?firmware_slot=0
135622025-04-30T06:44:20.238ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "99363aa7-7fca-4980-8e17-a6d785212274", "content-length": "178", "date": "Wed, 30 Apr 2025 06:44:20 GMT"} })
135632025-04-30T06:44:20.238ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1/component/stage0/caboose?firmware_slot=1
135642025-04-30T06:44:20.240ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "94743b4e-bae4-4407-8129-c008fe24b1b5", "content-length": "178", "date": "Wed, 30 Apr 2025 06:44:20 GMT"} })
135652025-04-30T06:44:20.240ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1/component/rot/cmpa
135662025-04-30T06:44:20.243ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "0e9c8595-9fe7-4179-883c-7991b992b430", "content-length": "702", "date": "Wed, 30 Apr 2025 06:44:20 GMT"} })
135672025-04-30T06:44:20.243ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1/component/rot/cfpa
135682025-04-30T06:44:20.244ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "a98c77e1-5211-4e34-8a56-93d2070856a5", "content-length": "727", "date": "Wed, 30 Apr 2025 06:44:20 GMT"} })
135692025-04-30T06:44:20.244ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1/component/rot/cfpa
135702025-04-30T06:44:20.245ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "3baf6b5d-f39f-416d-b8b3-06fa86e09391", "content-length": "729", "date": "Wed, 30 Apr 2025 06:44:20 GMT"} })
135712025-04-30T06:44:20.245ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1/component/rot/cfpa
135722025-04-30T06:44:20.246ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "d7e8642d-ac33-4233-a824-9650190ed287", "content-length": "728", "date": "Wed, 30 Apr 2025 06:44:20 GMT"} })
135732025-04-30T06:44:20.246ZTRACtest_utilization (BackgroundTasks): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet
135742025-04-30T06:44:20.246ZTRACtest_utilization (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
135752025-04-30T06:44:20.246ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
135762025-04-30T06:44:20.246ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
135772025-04-30T06:44:20.247ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
135782025-04-30T06:44:20.250ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
135792025-04-30T06:44:20.257ZDEBGtest_utilization (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet result = Ok(())
135802025-04-30T06:44:20.257ZTRACtest_utilization (BackgroundTasks): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet
135812025-04-30T06:44:20.257ZTRACtest_utilization (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
135822025-04-30T06:44:20.257ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
135832025-04-30T06:44:20.257ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
135842025-04-30T06:44:20.258ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
135852025-04-30T06:44:20.259ZTRACtest_utilization (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory
135862025-04-30T06:44:20.260ZTRACtest_utilization (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
135872025-04-30T06:44:20.260ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
135882025-04-30T06:44:20.260ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
135892025-04-30T06:44:20.260ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
135902025-04-30T06:44:20.260ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
135912025-04-30T06:44:20.263ZDEBGtest_utilization (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet result = Ok(())
135922025-04-30T06:44:20.263ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
135932025-04-30T06:44:20.263ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
135942025-04-30T06:44:20.264ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
135952025-04-30T06:44:20.264ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
135962025-04-30T06:44:20.273ZDEBGtest_utilization (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory result = Ok(())
135972025-04-30T06:44:20.273ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
135982025-04-30T06:44:20.273ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
135992025-04-30T06:44:20.273ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
136002025-04-30T06:44:20.368ZDEBGtest_utilization (BackgroundTasks): begin collection from Sled Agent background_task = inventory_collection sled_agent_url = http://[::1]:39365
136012025-04-30T06:44:20.368ZDEBGtest_utilization (BackgroundTasks): client request SledAgent = http://[::1]:39365 background_task = inventory_collection body = None method = GET uri = http://[::1]:39365/inventory
136022025-04-30T06:44:20.371ZINFOtest_utilization (dropshot): accepted connection local_addr = [::1]:39365 remote_addr = [::1]:52829 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
136032025-04-30T06:44:20.371ZDEBGtest_utilization (results-sink): inserting 1 samples into database collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
136042025-04-30T06:44:20.371ZDEBGtest_utilization (results-sink-cluster): inserting 1 samples into database collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
136052025-04-30T06:44:20.371ZDEBGtest_utilization (clickhouse-client): unrolling 1 total samples collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 4d1485f0-110e-4161-9a33-c4610d68be5d
136062025-04-30T06:44:20.371ZTRACtest_utilization (clickhouse-client): inserting data collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 4d1485f0-110e-4161-9a33-c4610d68be5d n_columns = 4 n_rows = 2 sql = INSERT INTO oximeter.fields_string FORMAT Native
136072025-04-30T06:44:20.372ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
136082025-04-30T06:44:20.372ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
136092025-04-30T06:44:20.373ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
136102025-04-30T06:44:20.375ZTRACtest_utilization (dropshot): incoming request local_addr = [::1]:39365 method = GET remote_addr = [::1]:52829 req_id = 6fb459e0-a775-4611-b771-986c9cf09e96 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /inventory
136112025-04-30T06:44:20.376ZINFOtest_utilization (dropshot): request completed latency_us = 600 local_addr = [::1]:39365 method = GET remote_addr = [::1]:52829 req_id = 6fb459e0-a775-4611-b771-986c9cf09e96 response_code = 200 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /inventory
136122025-04-30T06:44:20.376ZDEBGtest_utilization (clickhouse-client): inserted rows into table collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 4d1485f0-110e-4161-9a33-c4610d68be5d n_rows = 2 table_name = fields_string
136132025-04-30T06:44:20.376ZTRACtest_utilization (clickhouse-client): inserting data collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 4d1485f0-110e-4161-9a33-c4610d68be5d n_columns = 4 n_rows = 1 sql = INSERT INTO oximeter.measurements_i64 FORMAT Native
136142025-04-30T06:44:20.378ZDEBGtest_utilization (BackgroundTasks): client response SledAgent = http://[::1]:39365 background_task = inventory_collection result = Ok(Response { url: "http://[::1]:39365/inventory", status: 200, headers: {"content-type": "application/json", "x-request-id": "6fb459e0-a775-4611-b771-986c9cf09e96", "content-length": "3141", "date": "Wed, 30 Apr 2025 06:44:20 GMT"} })
136152025-04-30T06:44:20.380ZDEBGtest_utilization (BackgroundTasks): begin collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
136162025-04-30T06:44:20.380ZDEBGtest_utilization (BackgroundTasks): end collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
136172025-04-30T06:44:20.380ZDEBGtest_utilization (BackgroundTasks): finished collection background_task = inventory_collection
136182025-04-30T06:44:20.382ZTRACtest_utilization (BackgroundTasks): authorize begin action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory
136192025-04-30T06:44:20.382ZTRACtest_utilization (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
136202025-04-30T06:44:20.382ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
136212025-04-30T06:44:20.382ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
136222025-04-30T06:44:20.383ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
136232025-04-30T06:44:20.387ZDEBGtest_utilization (clickhouse-client): inserted rows into table collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 4d1485f0-110e-4161-9a33-c4610d68be5d n_rows = 1 table_name = measurements_i64
136242025-04-30T06:44:20.387ZTRACtest_utilization (results-sink): successfully inserted samples collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
136252025-04-30T06:44:20.388ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
136262025-04-30T06:44:20.393ZDEBGtest_utilization (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory result = Ok(())
136272025-04-30T06:44:20.393ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
136282025-04-30T06:44:20.394ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
136292025-04-30T06:44:20.400ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
136302025-04-30T06:44:20.426ZDEBGtest_utilization: attempt to fetch latest inventory collection result = Ok("found")
136312025-04-30T06:44:20.427ZWARNtest_utilization (datastore): Non-retryable transaction failure err = DatabaseError(UniqueViolation, "duplicate key value violates unique constraint \\"inv_caboose_pkey\\"") transaction = inventory_insert_collection
136322025-04-30T06:44:20.428ZWARNtest_utilization (BackgroundTasks): inventory collection failed background_task = inventory_collection error = failed to collect inventory: saving inventory to database: Internal Error: unexpected database error: duplicate key value violates unique constraint "inv_caboose_pkey"
136332025-04-30T06:44:20.428ZDEBGtest_utilization (BackgroundTasks): activation complete background_task = inventory_collection elapsed = 537.150828ms iteration = 36 status = {"error":"failed to collect inventory: saving inventory to database: Internal Error: unexpected database error: duplicate key value violates unique constraint \\"inv_caboose_pkey\\""}
136342025-04-30T06:44:20.428ZDEBGtest_utilization (BackgroundTasks): activating background_task = inventory_collection iteration = 37 reason = Signaled
136352025-04-30T06:44:20.428ZTRACtest_utilization (BackgroundTasks): authorize begin action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory
136362025-04-30T06:44:20.428ZTRACtest_utilization (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
136372025-04-30T06:44:20.428ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
136382025-04-30T06:44:20.428ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
136392025-04-30T06:44:20.429ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
136402025-04-30T06:44:20.431ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
136412025-04-30T06:44:20.436ZDEBGtest_utilization (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory result = Ok(())
136422025-04-30T06:44:20.437ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
136432025-04-30T06:44:20.437ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
136442025-04-30T06:44:20.437ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
136452025-04-30T06:44:20.440ZDEBGtest_utilization (BackgroundTasks): inventory_prune_one: nothing eligible for removal (too few) background_task = inventory_collection candidates = [(13d8b8d9-6c6b-47a6-9190-952544edb75b, 0)]
136462025-04-30T06:44:20.441ZTRACtest_utilization (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal
136472025-04-30T06:44:20.442ZTRACtest_utilization (dns): buffer buffer = Length: 67 (0x43) bytes\n0000: 33 e6 01 00 00 01 00 00 00 00 00 01 04 5f 6d 67 3............_mg\n0010: 73 04 5f 74 63 70 0d 63 6f 6e 74 72 6f 6c 2d 70 s._tcp.control-p\n0020: 6c 61 6e 65 05 6f 78 69 64 65 08 69 6e 74 65 72 lane.oxide.inter\n0030: 6e 61 6c 00 00 21 00 01 00 00 29 04 d0 00 00 00 nal..!....).....\n0040: 00 00 00 ... kind = dns peer_addr = [::1]:60630 req_id = 5518dda7-df86-4cf3-ac01-cf6823935070
136482025-04-30T06:44:20.442ZDEBGtest_utilization (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 13286,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_mgs._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_mgs._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 4,\n 95,\n 109,\n 103,\n 115,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:60630 req_id = 5518dda7-df86-4cf3-ac01-cf6823935070
136492025-04-30T06:44:20.442ZDEBGtest_utilization (store): query key key = _mgs._tcp
136502025-04-30T06:44:20.442ZDEBGtest_utilization (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
136512025-04-30T06:44:20.443ZDEBGtest_utilization (dns): dns response additional_records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal"), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] kind = dns peer_addr = [::1]:60630 query = LowerQuery { name: LowerName(Name("_mgs._tcp.control-plane.oxide.internal.")), original: Query { name: Name("_mgs._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN } } records = [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 48759, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal") })) }] req_id = 5518dda7-df86-4cf3-ac01-cf6823935070
136522025-04-30T06:44:20.443ZDEBGtest_utilization (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgs._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 48759, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }], valid_until: Instant { tv_sec: 4325, tv_nsec: 242568743 } })
136532025-04-30T06:44:20.443ZTRACtest_utilization (DnsResolver): lookup_service_targets: looking up SRV target
136542025-04-30T06:44:20.444ZTRACtest_utilization (dns): buffer buffer = Length: 108 (0x6c) bytes\n0000: 3f 09 01 00 00 01 00 00 00 00 00 01 2d 64 65 6e ?...........-den\n0010: 64 72 69 74 65 2d 62 36 64 36 35 33 34 31 2d 31 drite-b6d65341-1\n0020: 36 37 63 2d 34 31 64 66 2d 39 62 35 63 2d 34 31 67c-41df-9b5c-41\n0030: 63 64 65 64 39 39 63 32 32 39 04 68 6f 73 74 0d cded99c229.host.\n0040: 63 6f 6e 74 72 6f 6c 2d 70 6c 61 6e 65 05 6f 78 control-plane.ox\n0050: 69 64 65 08 69 6e 74 65 72 6e 61 6c 00 00 1c 00 ide.internal....\n0060: 01 00 00 29 04 d0 00 00 00 00 00 00 ...)........ kind = dns peer_addr = [::1]:55197 req_id = 598fadbd-5b69-4feb-94e6-4785a38406cf
136552025-04-30T06:44:20.444ZDEBGtest_utilization (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 16137,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n query_type: AAAA,\n query_class: IN,\n },\n },\n original: [\n 45,\n 100,\n 101,\n 110,\n 100,\n 114,\n 105,\n 116,\n 101,\n 45,\n 98,\n 54,\n 100,\n 54,\n 53,\n 51,\n 52,\n 49,\n 45,\n 49,\n 54,\n 55,\n 99,\n 45,\n 52,\n 49,\n 100,\n 102,\n 45,\n 57,\n 98,\n 53,\n 99,\n 45,\n 52,\n 49,\n 99,\n 100,\n 101,\n 100,\n 57,\n 57,\n 99,\n 50,\n 50,\n 57,\n 4,\n 104,\n 111,\n 115,\n 116,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 28,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:55197 req_id = 598fadbd-5b69-4feb-94e6-4785a38406cf
136562025-04-30T06:44:20.444ZDEBGtest_utilization (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
136572025-04-30T06:44:20.444ZDEBGtest_utilization (dns): dns response additional_records = [] kind = dns peer_addr = [::1]:55197 query = LowerQuery { name: LowerName(Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.")), original: Query { name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), query_type: AAAA, query_class: IN } } records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] req_id = 598fadbd-5b69-4feb-94e6-4785a38406cf
136582025-04-30T06:44:20.520ZTRACtest_utilization (DnsResolver): lookup_all_socket_v6 srv dns_name = _clickhouse-admin-keeper._tcp.control-plane.oxide.internal
136592025-04-30T06:44:20.520ZTRACtest_utilization (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory
136602025-04-30T06:44:20.520ZTRACtest_utilization (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
136612025-04-30T06:44:20.520ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
136622025-04-30T06:44:20.520ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
136632025-04-30T06:44:20.522ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
136642025-04-30T06:44:20.526ZWARNtest_utilization (results-sink-cluster): failed to insert some results into metric DB collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 error = Failed to check out connection to database: No backends found for this service file = oximeter/collector/src/results_sink.rs:92
136652025-04-30T06:44:20.527ZTRACtest_utilization (dns): buffer buffer = Length: 87 (0x57) bytes\n0000: 69 68 01 00 00 01 00 00 00 00 00 01 18 5f 63 6c ih..........._cl\n0010: 69 63 6b 68 6f 75 73 65 2d 61 64 6d 69 6e 2d 6b ickhouse-admin-k\n0020: 65 65 70 65 72 04 5f 74 63 70 0d 63 6f 6e 74 72 eeper._tcp.contr\n0030: 6f 6c 2d 70 6c 61 6e 65 05 6f 78 69 64 65 08 69 ol-plane.oxide.i\n0040: 6e 74 65 72 6e 61 6c 00 00 21 00 01 00 00 29 04 nternal..!....).\n0050: d0 00 00 00 00 00 00 ....... kind = dns peer_addr = [::1]:53324 req_id = 8f98c784-5c2e-418a-82ee-18a7124fa208
136662025-04-30T06:44:20.527ZDEBGtest_utilization (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 26984,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 24,\n 95,\n 99,\n 108,\n 105,\n 99,\n 107,\n 104,\n 111,\n 117,\n 115,\n 101,\n 45,\n 97,\n 100,\n 109,\n 105,\n 110,\n 45,\n 107,\n 101,\n 101,\n 112,\n 101,\n 114,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:53324 req_id = 8f98c784-5c2e-418a-82ee-18a7124fa208
136672025-04-30T06:44:20.527ZDEBGtest_utilization (store): query key key = _clickhouse-admin-keeper._tcp
136682025-04-30T06:44:20.527ZERROtest_utilization (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 26984, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 1, }, query: WireQuery { query: LowerQuery { name: LowerName( Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), ), original: Query { name: Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN, }, }, original: [ 24, 95, 99, 108, 105, 99, 107, 104, 111, 117, 115, 101, 45, 97, 100, 109, 105, 110, 45, 107, 101, 101, 112, 101, 114, 4, 95, 116, 99, 112, 13, 99, 111, 110, 116, 114, 111, 108, 45, 112, 108, 97, 110, 101, 5, 111, 120, 105, 100, 101, 8, 105, 110, 116, 101, 114, 110, 97, 108, 0, 0, 33, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: Some( Edns { rcode_high: 0, version: 0, dnssec_ok: false, max_payload: 1232, options: OPT { options: {}, }, }, ), } NXDOMAIN: no records found for name: "_clickhouse-admin-keeper._tcp.control-plane.oxide.internal." kind = dns peer_addr = [::1]:53324 req_id = 8f98c784-5c2e-418a-82ee-18a7124fa208
136692025-04-30T06:44:21.038ZDEBGtest_utilization (BackgroundTasks): begin collection background_task = inventory_collection
136702025-04-30T06:44:21.038ZDEBGtest_utilization (BackgroundTasks): begin collection from MGS background_task = inventory_collection mgs_url = http://[::1]:48759
136712025-04-30T06:44:21.038ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/ignition
136722025-04-30T06:44:21.038ZINFOtest_utilization (dropshot): accepted connection kind = pantry local_addr = [::1]:45558 remote_addr = [::1]:47579 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
136732025-04-30T06:44:21.039ZINFOtest_utilization (dropshot): accepted connection kind = pantry local_addr = [::1]:45558 remote_addr = [::1]:38961 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
136742025-04-30T06:44:21.039ZINFOtest_utilization (dropshot): accepted connection kind = pantry local_addr = [::1]:45558 remote_addr = [::1]:43699 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
136752025-04-30T06:44:21.039ZINFOtest_utilization (dropshot): accepted connection kind = pantry local_addr = [::1]:45558 remote_addr = [::1]:60250 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
136762025-04-30T06:44:21.039ZINFOtest_utilization (dropshot): accepted connection kind = pantry local_addr = [::1]:45558 remote_addr = [::1]:40544 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
136772025-04-30T06:44:21.039ZINFOtest_utilization (dropshot): accepted connection kind = pantry local_addr = [::1]:45558 remote_addr = [::1]:33642 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
136782025-04-30T06:44:21.040ZINFOtest_utilization (dropshot): accepted connection kind = pantry local_addr = [::1]:45558 remote_addr = [::1]:43779 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
136792025-04-30T06:44:21.041ZTRACtest_utilization (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal
136802025-04-30T06:44:21.042ZTRACtest_utilization (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal
136812025-04-30T06:44:21.042ZDEBGtest_utilization (collection-task): sent timer-based collection request to the collection task collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
136822025-04-30T06:44:21.045ZDEBGtest_utilization (collection-task): collecting from producer collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
136832025-04-30T06:44:21.046ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
136842025-04-30T06:44:21.054ZDEBGtest_utilization (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory result = Ok(())
136852025-04-30T06:44:21.054ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
136862025-04-30T06:44:21.055ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
136872025-04-30T06:44:21.055ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
136882025-04-30T06:44:21.061ZTRACtest_utilization (dropshot): incoming request kind = pantry local_addr = [::1]:45558 method = GET remote_addr = [::1]:47579 req_id = b761d8f0-68da-46fc-8162-280ef2a9d77b sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /crucible/pantry/0
136892025-04-30T06:44:21.061ZTRACtest_utilization (dropshot): incoming request kind = pantry local_addr = [::1]:45558 method = GET remote_addr = [::1]:38961 req_id = 884cf470-34d0-4f59-acd5-20fb9985c3b5 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /crucible/pantry/0
136902025-04-30T06:44:21.062ZTRACtest_utilization (dropshot): incoming request kind = pantry local_addr = [::1]:45558 method = GET remote_addr = [::1]:43699 req_id = 6b8b1cca-4f23-41cf-8386-597bedeb9fc1 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /crucible/pantry/0
136912025-04-30T06:44:21.063ZTRACtest_utilization (dropshot): incoming request kind = pantry local_addr = [::1]:45558 method = GET remote_addr = [::1]:60250 req_id = 18438caf-7116-44ce-b63d-1a2b11afff99 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /crucible/pantry/0
136922025-04-30T06:44:21.063ZTRACtest_utilization (dropshot): incoming request kind = pantry local_addr = [::1]:45558 method = GET remote_addr = [::1]:40544 req_id = f34d9803-967e-4564-a4bc-bffc4ef03397 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /crucible/pantry/0
136932025-04-30T06:44:21.063ZTRACtest_utilization (dropshot): incoming request kind = pantry local_addr = [::1]:45558 method = GET remote_addr = [::1]:33642 req_id = d27ed28f-ac7d-4287-ac49-e11cd0981cc1 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /crucible/pantry/0
136942025-04-30T06:44:21.063ZTRACtest_utilization (dropshot): incoming request kind = pantry local_addr = [::1]:45558 method = GET remote_addr = [::1]:43779 req_id = c7f1ec53-2b03-4529-a51a-a24ac8729ffc sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /crucible/pantry/0
136952025-04-30T06:44:21.065ZTRACtest_utilization (dns): buffer buffer = Length: 72 (0x48) bytes\n0000: f1 12 01 00 00 01 00 00 00 00 00 01 09 5f 64 65 ............._de\n0010: 6e 64 72 69 74 65 04 5f 74 63 70 0d 63 6f 6e 74 ndrite._tcp.cont\n0020: 72 6f 6c 2d 70 6c 61 6e 65 05 6f 78 69 64 65 08 rol-plane.oxide.\n0030: 69 6e 74 65 72 6e 61 6c 00 00 21 00 01 00 00 29 internal..!....)\n0040: 04 d0 00 00 00 00 00 00 ........ kind = dns peer_addr = [::1]:54310 req_id = 5e65018b-cbad-433a-b920-d2ed4ce2fc64
136962025-04-30T06:44:21.065ZDEBGtest_utilization (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 61714,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_dendrite._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_dendrite._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 9,\n 95,\n 100,\n 101,\n 110,\n 100,\n 114,\n 105,\n 116,\n 101,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:54310 req_id = 5e65018b-cbad-433a-b920-d2ed4ce2fc64
136972025-04-30T06:44:21.066ZDEBGtest_utilization (store): query key key = _dendrite._tcp
136982025-04-30T06:44:21.066ZDEBGtest_utilization (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
136992025-04-30T06:44:21.066ZDEBGtest_utilization (dns): dns response additional_records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal"), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] kind = dns peer_addr = [::1]:54310 query = LowerQuery { name: LowerName(Name("_dendrite._tcp.control-plane.oxide.internal.")), original: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN } } records = [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 47974, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal") })) }] req_id = 5e65018b-cbad-433a-b920-d2ed4ce2fc64
137002025-04-30T06:44:21.066ZTRACtest_utilization (dns): buffer buffer = Length: 72 (0x48) bytes\n0000: d5 22 01 00 00 01 00 00 00 00 00 01 09 5f 64 65 ."..........._de\n0010: 6e 64 72 69 74 65 04 5f 74 63 70 0d 63 6f 6e 74 ndrite._tcp.cont\n0020: 72 6f 6c 2d 70 6c 61 6e 65 05 6f 78 69 64 65 08 rol-plane.oxide.\n0030: 69 6e 74 65 72 6e 61 6c 00 00 21 00 01 00 00 29 internal..!....)\n0040: 04 d0 00 00 00 00 00 00 ........ kind = dns peer_addr = [::1]:55946 req_id = 033318ba-8b60-44d9-b115-91c929e2069b
137012025-04-30T06:44:21.066ZDEBGtest_utilization (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 54562,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_dendrite._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_dendrite._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 9,\n 95,\n 100,\n 101,\n 110,\n 100,\n 114,\n 105,\n 116,\n 101,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:55946 req_id = 033318ba-8b60-44d9-b115-91c929e2069b
137022025-04-30T06:44:21.066ZDEBGtest_utilization (store): query key key = _dendrite._tcp
137032025-04-30T06:44:21.066ZDEBGtest_utilization (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
137042025-04-30T06:44:21.066ZDEBGtest_utilization (dns): dns response additional_records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal"), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] kind = dns peer_addr = [::1]:55946 query = LowerQuery { name: LowerName(Name("_dendrite._tcp.control-plane.oxide.internal.")), original: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN } } records = [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 47974, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal") })) }] req_id = 033318ba-8b60-44d9-b115-91c929e2069b
137052025-04-30T06:44:21.070ZINFOtest_utilization (dropshot): request completed kind = pantry latency_us = 8391 local_addr = [::1]:45558 method = GET remote_addr = [::1]:47579 req_id = b761d8f0-68da-46fc-8162-280ef2a9d77b response_code = 200 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /crucible/pantry/0
137062025-04-30T06:44:21.070ZINFOtest_utilization (dropshot): request completed kind = pantry latency_us = 8499 local_addr = [::1]:45558 method = GET remote_addr = [::1]:38961 req_id = 884cf470-34d0-4f59-acd5-20fb9985c3b5 response_code = 200 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /crucible/pantry/0
137072025-04-30T06:44:21.070ZINFOtest_utilization (dropshot): request completed kind = pantry latency_us = 8696 local_addr = [::1]:45558 method = GET remote_addr = [::1]:43699 req_id = 6b8b1cca-4f23-41cf-8386-597bedeb9fc1 response_code = 200 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /crucible/pantry/0
137082025-04-30T06:44:21.070ZINFOtest_utilization (dropshot): request completed kind = pantry latency_us = 7616 local_addr = [::1]:45558 method = GET remote_addr = [::1]:60250 req_id = 18438caf-7116-44ce-b63d-1a2b11afff99 response_code = 200 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /crucible/pantry/0
137092025-04-30T06:44:21.070ZINFOtest_utilization (dropshot): request completed kind = pantry latency_us = 7541 local_addr = [::1]:45558 method = GET remote_addr = [::1]:40544 req_id = f34d9803-967e-4564-a4bc-bffc4ef03397 response_code = 200 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /crucible/pantry/0
137102025-04-30T06:44:21.070ZINFOtest_utilization (dropshot): request completed kind = pantry latency_us = 7328 local_addr = [::1]:45558 method = GET remote_addr = [::1]:33642 req_id = d27ed28f-ac7d-4287-ac49-e11cd0981cc1 response_code = 200 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /crucible/pantry/0
137112025-04-30T06:44:21.070ZINFOtest_utilization (dropshot): request completed kind = pantry latency_us = 7200 local_addr = [::1]:45558 method = GET remote_addr = [::1]:43779 req_id = c7f1ec53-2b03-4529-a51a-a24ac8729ffc response_code = 200 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /crucible/pantry/0
137122025-04-30T06:44:21.074ZDEBGtest_utilization (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 47974, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }], valid_until: Instant { tv_sec: 4325, tv_nsec: 873033132 } })
137132025-04-30T06:44:21.074ZTRACtest_utilization (DnsResolver): lookup_service_targets: looking up SRV target
137142025-04-30T06:44:21.075ZDEBGtest_utilization (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 47974, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }], valid_until: Instant { tv_sec: 4325, tv_nsec: 874026882 } })
137152025-04-30T06:44:21.075ZTRACtest_utilization (DnsResolver): lookup_service_targets: looking up SRV target
137162025-04-30T06:44:21.076ZTRACtest_utilization (collection-task): sent collection request to producer collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
137172025-04-30T06:44:21.078ZDEBGtest_utilization (collection-task): collected results from producer collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 n_results = 1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
137182025-04-30T06:44:21.078ZTRACtest_utilization (collection-task): forwarded results to main collection loop collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
137192025-04-30T06:44:21.078ZTRACtest_utilization (collection-task): top of inner collection loop, waiting for next request collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
137202025-04-30T06:44:21.078ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
137212025-04-30T06:44:21.078ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
137222025-04-30T06:44:21.079ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
137232025-04-30T06:44:21.086ZTRACtest_utilization (dns): buffer buffer = Length: 108 (0x6c) bytes\n0000: 6a fe 01 00 00 01 00 00 00 00 00 01 2d 64 65 6e j...........-den\n0010: 64 72 69 74 65 2d 62 36 64 36 35 33 34 31 2d 31 drite-b6d65341-1\n0020: 36 37 63 2d 34 31 64 66 2d 39 62 35 63 2d 34 31 67c-41df-9b5c-41\n0030: 63 64 65 64 39 39 63 32 32 39 04 68 6f 73 74 0d cded99c229.host.\n0040: 63 6f 6e 74 72 6f 6c 2d 70 6c 61 6e 65 05 6f 78 control-plane.ox\n0050: 69 64 65 08 69 6e 74 65 72 6e 61 6c 00 00 1c 00 ide.internal....\n0060: 01 00 00 29 04 d0 00 00 00 00 00 00 ...)........ kind = dns peer_addr = [::1]:55919 req_id = 658f42e0-da71-4691-b80a-8a23efe8c8fc
137242025-04-30T06:44:21.086ZDEBGtest_utilization (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 27390,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n query_type: AAAA,\n query_class: IN,\n },\n },\n original: [\n 45,\n 100,\n 101,\n 110,\n 100,\n 114,\n 105,\n 116,\n 101,\n 45,\n 98,\n 54,\n 100,\n 54,\n 53,\n 51,\n 52,\n 49,\n 45,\n 49,\n 54,\n 55,\n 99,\n 45,\n 52,\n 49,\n 100,\n 102,\n 45,\n 57,\n 98,\n 53,\n 99,\n 45,\n 52,\n 49,\n 99,\n 100,\n 101,\n 100,\n 57,\n 57,\n 99,\n 50,\n 50,\n 57,\n 4,\n 104,\n 111,\n 115,\n 116,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 28,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:55919 req_id = 658f42e0-da71-4691-b80a-8a23efe8c8fc
137252025-04-30T06:44:21.086ZDEBGtest_utilization (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
137262025-04-30T06:44:21.086ZDEBGtest_utilization (dns): dns response additional_records = [] kind = dns peer_addr = [::1]:55919 query = LowerQuery { name: LowerName(Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.")), original: Query { name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), query_type: AAAA, query_class: IN } } records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] req_id = 658f42e0-da71-4691-b80a-8a23efe8c8fc
137272025-04-30T06:44:21.086ZTRACtest_utilization (dns): buffer buffer = Length: 108 (0x6c) bytes\n0000: eb 77 01 00 00 01 00 00 00 00 00 01 2d 64 65 6e .w..........-den\n0010: 64 72 69 74 65 2d 62 36 64 36 35 33 34 31 2d 31 drite-b6d65341-1\n0020: 36 37 63 2d 34 31 64 66 2d 39 62 35 63 2d 34 31 67c-41df-9b5c-41\n0030: 63 64 65 64 39 39 63 32 32 39 04 68 6f 73 74 0d cded99c229.host.\n0040: 63 6f 6e 74 72 6f 6c 2d 70 6c 61 6e 65 05 6f 78 control-plane.ox\n0050: 69 64 65 08 69 6e 74 65 72 6e 61 6c 00 00 1c 00 ide.internal....\n0060: 01 00 00 29 04 d0 00 00 00 00 00 00 ...)........ kind = dns peer_addr = [::1]:63246 req_id = c3a51e88-1a07-4be4-960d-4041d12b6f58
137282025-04-30T06:44:21.086ZDEBGtest_utilization (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 60279,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n query_type: AAAA,\n query_class: IN,\n },\n },\n original: [\n 45,\n 100,\n 101,\n 110,\n 100,\n 114,\n 105,\n 116,\n 101,\n 45,\n 98,\n 54,\n 100,\n 54,\n 53,\n 51,\n 52,\n 49,\n 45,\n 49,\n 54,\n 55,\n 99,\n 45,\n 52,\n 49,\n 100,\n 102,\n 45,\n 57,\n 98,\n 53,\n 99,\n 45,\n 52,\n 49,\n 99,\n 100,\n 101,\n 100,\n 57,\n 57,\n 99,\n 50,\n 50,\n 57,\n 4,\n 104,\n 111,\n 115,\n 116,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 28,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:63246 req_id = c3a51e88-1a07-4be4-960d-4041d12b6f58
137292025-04-30T06:44:21.086ZDEBGtest_utilization (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
137302025-04-30T06:44:21.086ZDEBGtest_utilization (dns): dns response additional_records = [] kind = dns peer_addr = [::1]:63246 query = LowerQuery { name: LowerName(Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.")), original: Query { name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), query_type: AAAA, query_class: IN } } records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] req_id = c3a51e88-1a07-4be4-960d-4041d12b6f58
137312025-04-30T06:44:21.088ZINFOtest_utilization (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_v4_garbage_collector
137322025-04-30T06:44:21.154ZINFOtest_utilization (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_v4_garbage_collector zone_address = ::1
137332025-04-30T06:44:21.154ZDEBGtest_utilization (MgsClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[::1]:12225/local/switch-id
137342025-04-30T06:44:21.154ZDEBGtest_utilization (MgsClient): client response background_task = nat_v4_garbage_collector result = Err(reqwest::Error { kind: Request, url: "http://[::1]:12225/local/switch-id", source: hyper_util::client::legacy::Error(Connect, ConnectError("tcp connect error", Os { code: 146, kind: ConnectionRefused, message: "Connection refused" })) })
137352025-04-30T06:44:21.154ZWARNtest_utilization (BackgroundTasks): failed to identify switch slot for dendrite background_task = nat_v4_garbage_collector reason = Communication Error: error sending request for url (http://[::1]:12225/local/switch-id): client error (Connect): tcp connect error: Connection refused (os error 146): Connection refused (os error 146) zone_address = ::1
137362025-04-30T06:44:21.157ZWARNtest_utilization (BackgroundTasks): Failed to map switch zone addr: Communication Error: error sending request for url (http://[::1]:12225/local/switch-id), retrying background_task = nat_v4_garbage_collector
137372025-04-30T06:44:21.157ZINFOtest_utilization (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager
137382025-04-30T06:44:21.224ZINFOtest_utilization (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager zone_address = ::1
137392025-04-30T06:44:21.224ZDEBGtest_utilization (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[::1]:12225/local/switch-id
137402025-04-30T06:44:21.224ZDEBGtest_utilization (MgsClient): client response background_task = bfd_manager result = Err(reqwest::Error { kind: Request, url: "http://[::1]:12225/local/switch-id", source: hyper_util::client::legacy::Error(Connect, ConnectError("tcp connect error", Os { code: 146, kind: ConnectionRefused, message: "Connection refused" })) })
137412025-04-30T06:44:21.224ZWARNtest_utilization (BackgroundTasks): failed to identify switch slot for dendrite background_task = bfd_manager reason = Communication Error: error sending request for url (http://[::1]:12225/local/switch-id): client error (Connect): tcp connect error: Connection refused (os error 146): Connection refused (os error 146) zone_address = ::1
137422025-04-30T06:44:21.226ZWARNtest_utilization (BackgroundTasks): Failed to map switch zone addr: Communication Error: error sending request for url (http://[::1]:12225/local/switch-id), retrying background_task = bfd_manager
137432025-04-30T06:44:21.230ZTRACtest_utilization (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal
137442025-04-30T06:44:21.230ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/ignition", status: 200, headers: {"content-type": "application/json", "x-request-id": "efee27a3-8618-4895-8fd7-a4fab2d184d8", "content-length": "835", "date": "Wed, 30 Apr 2025 06:44:21 GMT"} })
137452025-04-30T06:44:21.230ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0
137462025-04-30T06:44:21.238ZTRACtest_utilization (dns): buffer buffer = Length: 72 (0x48) bytes\n0000: df e7 01 00 00 01 00 00 00 00 00 01 09 5f 64 65 ............._de\n0010: 6e 64 72 69 74 65 04 5f 74 63 70 0d 63 6f 6e 74 ndrite._tcp.cont\n0020: 72 6f 6c 2d 70 6c 61 6e 65 05 6f 78 69 64 65 08 rol-plane.oxide.\n0030: 69 6e 74 65 72 6e 61 6c 00 00 21 00 01 00 00 29 internal..!....)\n0040: 04 d0 00 00 00 00 00 00 ........ kind = dns peer_addr = [::1]:54370 req_id = 060b72c4-5c6c-4cfc-a313-9abd00dd3129
137472025-04-30T06:44:21.238ZDEBGtest_utilization (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 57319,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_dendrite._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_dendrite._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 9,\n 95,\n 100,\n 101,\n 110,\n 100,\n 114,\n 105,\n 116,\n 101,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:54370 req_id = 060b72c4-5c6c-4cfc-a313-9abd00dd3129
137482025-04-30T06:44:21.239ZDEBGtest_utilization (store): query key key = _dendrite._tcp
137492025-04-30T06:44:21.240ZDEBGtest_utilization (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
137502025-04-30T06:44:21.240ZDEBGtest_utilization (dns): dns response additional_records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal"), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] kind = dns peer_addr = [::1]:54370 query = LowerQuery { name: LowerName(Name("_dendrite._tcp.control-plane.oxide.internal.")), original: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN } } records = [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 47974, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal") })) }] req_id = 060b72c4-5c6c-4cfc-a313-9abd00dd3129
137512025-04-30T06:44:21.242ZDEBGtest_utilization (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 47974, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }], valid_until: Instant { tv_sec: 4326, tv_nsec: 40799064 } })
137522025-04-30T06:44:21.242ZTRACtest_utilization (DnsResolver): lookup_service_targets: looking up SRV target
137532025-04-30T06:44:21.253ZTRACtest_utilization (dns): buffer buffer = Length: 108 (0x6c) bytes\n0000: 63 47 01 00 00 01 00 00 00 00 00 01 2d 64 65 6e cG..........-den\n0010: 64 72 69 74 65 2d 62 36 64 36 35 33 34 31 2d 31 drite-b6d65341-1\n0020: 36 37 63 2d 34 31 64 66 2d 39 62 35 63 2d 34 31 67c-41df-9b5c-41\n0030: 63 64 65 64 39 39 63 32 32 39 04 68 6f 73 74 0d cded99c229.host.\n0040: 63 6f 6e 74 72 6f 6c 2d 70 6c 61 6e 65 05 6f 78 control-plane.ox\n0050: 69 64 65 08 69 6e 74 65 72 6e 61 6c 00 00 1c 00 ide.internal....\n0060: 01 00 00 29 04 d0 00 00 00 00 00 00 ...)........ kind = dns peer_addr = [::1]:60269 req_id = 35d31150-c8a4-4484-9684-ef5747768697
137542025-04-30T06:44:21.253ZDEBGtest_utilization (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 25415,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n query_type: AAAA,\n query_class: IN,\n },\n },\n original: [\n 45,\n 100,\n 101,\n 110,\n 100,\n 114,\n 105,\n 116,\n 101,\n 45,\n 98,\n 54,\n 100,\n 54,\n 53,\n 51,\n 52,\n 49,\n 45,\n 49,\n 54,\n 55,\n 99,\n 45,\n 52,\n 49,\n 100,\n 102,\n 45,\n 57,\n 98,\n 53,\n 99,\n 45,\n 52,\n 49,\n 99,\n 100,\n 101,\n 100,\n 57,\n 57,\n 99,\n 50,\n 50,\n 57,\n 4,\n 104,\n 111,\n 115,\n 116,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 28,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:60269 req_id = 35d31150-c8a4-4484-9684-ef5747768697
137552025-04-30T06:44:21.253ZDEBGtest_utilization (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
137562025-04-30T06:44:21.253ZDEBGtest_utilization (dns): dns response additional_records = [] kind = dns peer_addr = [::1]:60269 query = LowerQuery { name: LowerName(Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.")), original: Query { name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), query_type: AAAA, query_class: IN } } records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] req_id = 35d31150-c8a4-4484-9684-ef5747768697
137572025-04-30T06:44:21.254ZINFOtest_utilization (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager rack_id = c19a698f-c6f9-4a17-ae30-20d711b8f7dc
137582025-04-30T06:44:21.326ZINFOtest_utilization (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager rack_id = c19a698f-c6f9-4a17-ae30-20d711b8f7dc zone_address = ::1
137592025-04-30T06:44:21.327ZDEBGtest_utilization (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = c19a698f-c6f9-4a17-ae30-20d711b8f7dc uri = http://[::1]:12225/local/switch-id
137602025-04-30T06:44:21.327ZDEBGtest_utilization (MgsClient): client response background_task = switch_port_config_manager rack_id = c19a698f-c6f9-4a17-ae30-20d711b8f7dc result = Err(reqwest::Error { kind: Request, url: "http://[::1]:12225/local/switch-id", source: hyper_util::client::legacy::Error(Connect, ConnectError("tcp connect error", Os { code: 146, kind: ConnectionRefused, message: "Connection refused" })) })
137612025-04-30T06:44:21.327ZWARNtest_utilization (BackgroundTasks): failed to identify switch slot for dendrite background_task = switch_port_config_manager rack_id = c19a698f-c6f9-4a17-ae30-20d711b8f7dc reason = Communication Error: error sending request for url (http://[::1]:12225/local/switch-id): client error (Connect): tcp connect error: Connection refused (os error 146): Connection refused (os error 146) zone_address = ::1
137622025-04-30T06:44:21.330ZWARNtest_utilization (BackgroundTasks): Failed to map switch zone addr: Communication Error: error sending request for url (http://[::1]:12225/local/switch-id), retrying background_task = switch_port_config_manager rack_id = c19a698f-c6f9-4a17-ae30-20d711b8f7dc
137632025-04-30T06:44:21.337ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "6703c063-26bd-4436-abdf-13af8e4d8ad3", "content-length": "734", "date": "Wed, 30 Apr 2025 06:44:21 GMT"} })
137642025-04-30T06:44:21.337ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0/component/sp/caboose?firmware_slot=0
137652025-04-30T06:44:21.357ZDEBGtest_utilization (results-sink): inserting 1 samples into database collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
137662025-04-30T06:44:21.357ZDEBGtest_utilization (results-sink-cluster): inserting 1 samples into database collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
137672025-04-30T06:44:21.357ZDEBGtest_utilization (clickhouse-client): unrolling 1 total samples collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 4d1485f0-110e-4161-9a33-c4610d68be5d
137682025-04-30T06:44:21.359ZTRACtest_utilization (clickhouse-client): inserting data collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 4d1485f0-110e-4161-9a33-c4610d68be5d n_columns = 4 n_rows = 2 sql = INSERT INTO oximeter.fields_string FORMAT Native
137692025-04-30T06:44:21.360ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "db567eb7-9add-4470-a542-6ac7d5b75b34", "content-length": "111", "date": "Wed, 30 Apr 2025 06:44:21 GMT"} })
137702025-04-30T06:44:21.360ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0/component/sp/caboose?firmware_slot=1
137712025-04-30T06:44:21.365ZDEBGtest_utilization (clickhouse-client): inserted rows into table collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 4d1485f0-110e-4161-9a33-c4610d68be5d n_rows = 2 table_name = fields_string
137722025-04-30T06:44:21.365ZTRACtest_utilization (clickhouse-client): inserting data collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 4d1485f0-110e-4161-9a33-c4610d68be5d n_columns = 4 n_rows = 1 sql = INSERT INTO oximeter.measurements_i64 FORMAT Native
137732025-04-30T06:44:21.369ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "992e781d-fe32-46e2-b8a6-4eb8e53a1f83", "content-length": "111", "date": "Wed, 30 Apr 2025 06:44:21 GMT"} })
137742025-04-30T06:44:21.370ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0/component/rot/caboose?firmware_slot=0
137752025-04-30T06:44:21.374ZDEBGtest_utilization (clickhouse-client): inserted rows into table collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 4d1485f0-110e-4161-9a33-c4610d68be5d n_rows = 1 table_name = measurements_i64
137762025-04-30T06:44:21.375ZTRACtest_utilization (results-sink): successfully inserted samples collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
137772025-04-30T06:44:21.375ZDEBGtest_utilization: attempt to fetch latest inventory collection result = Ok("found")
137782025-04-30T06:44:21.380ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "f92f250c-cd62-45d5-8414-27a239576602", "content-length": "170", "date": "Wed, 30 Apr 2025 06:44:21 GMT"} })
137792025-04-30T06:44:21.380ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0/component/rot/caboose?firmware_slot=1
137802025-04-30T06:44:21.383ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "4f55b6f6-c256-4b10-9cde-67a1f1fde66a", "content-length": "170", "date": "Wed, 30 Apr 2025 06:44:21 GMT"} })
137812025-04-30T06:44:21.383ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0/component/stage0/caboose?firmware_slot=0
137822025-04-30T06:44:21.386ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "061ec7e9-d56a-44ec-b4e3-752815f9ae66", "content-length": "179", "date": "Wed, 30 Apr 2025 06:44:21 GMT"} })
137832025-04-30T06:44:21.386ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0/component/stage0/caboose?firmware_slot=1
137842025-04-30T06:44:21.388ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "7d6bc63b-2ef6-4331-9005-3f9bff8a26a3", "content-length": "179", "date": "Wed, 30 Apr 2025 06:44:21 GMT"} })
137852025-04-30T06:44:21.389ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0/component/rot/cmpa
137862025-04-30T06:44:21.391ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "9c0c3844-2bf0-4769-8334-538dd5d2900f", "content-length": "702", "date": "Wed, 30 Apr 2025 06:44:21 GMT"} })
137872025-04-30T06:44:21.393ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0/component/rot/cfpa
137882025-04-30T06:44:21.395ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "8fe119f3-ac9d-40d3-9b8c-da8130b2c399", "content-length": "727", "date": "Wed, 30 Apr 2025 06:44:21 GMT"} })
137892025-04-30T06:44:21.395ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0/component/rot/cfpa
137902025-04-30T06:44:21.396ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "1b65f457-00f8-49bc-85f0-54a89129a524", "content-length": "729", "date": "Wed, 30 Apr 2025 06:44:21 GMT"} })
137912025-04-30T06:44:21.396ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0/component/rot/cfpa
137922025-04-30T06:44:21.400ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "a3befd38-f560-480e-958f-475b5c391c87", "content-length": "728", "date": "Wed, 30 Apr 2025 06:44:21 GMT"} })
137932025-04-30T06:44:21.400ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1
137942025-04-30T06:44:21.401ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1", status: 200, headers: {"content-type": "application/json", "x-request-id": "fa22700c-4f4d-4838-8f06-c0db87784bb8", "content-length": "734", "date": "Wed, 30 Apr 2025 06:44:21 GMT"} })
137952025-04-30T06:44:21.401ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1/component/sp/caboose?firmware_slot=0
137962025-04-30T06:44:21.405ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "73e3d2bb-d408-4fe0-af1e-6b8fd34cd90c", "content-length": "111", "date": "Wed, 30 Apr 2025 06:44:21 GMT"} })
137972025-04-30T06:44:21.406ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1/component/sp/caboose?firmware_slot=1
137982025-04-30T06:44:21.408ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "fd4f6564-f050-4f05-9e93-24a140a26190", "content-length": "111", "date": "Wed, 30 Apr 2025 06:44:21 GMT"} })
137992025-04-30T06:44:21.408ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1/component/rot/caboose?firmware_slot=0
138002025-04-30T06:44:21.410ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "4ba439b5-eee4-40b8-a8e1-ef322a1c0e5c", "content-length": "170", "date": "Wed, 30 Apr 2025 06:44:21 GMT"} })
138012025-04-30T06:44:21.410ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1/component/rot/caboose?firmware_slot=1
138022025-04-30T06:44:21.413ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "c7a92cd8-8945-4abb-9e1b-2b07bb6daff5", "content-length": "170", "date": "Wed, 30 Apr 2025 06:44:21 GMT"} })
138032025-04-30T06:44:21.414ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1/component/stage0/caboose?firmware_slot=0
138042025-04-30T06:44:21.416ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "40536b5e-ebae-4ff9-9b48-03cf9345e993", "content-length": "179", "date": "Wed, 30 Apr 2025 06:44:21 GMT"} })
138052025-04-30T06:44:21.417ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1/component/stage0/caboose?firmware_slot=1
138062025-04-30T06:44:21.420ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "cb38f148-676b-4786-89e0-d48531acdb9e", "content-length": "179", "date": "Wed, 30 Apr 2025 06:44:21 GMT"} })
138072025-04-30T06:44:21.420ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1/component/rot/cmpa
138082025-04-30T06:44:21.423ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "fb28bfd8-749a-463c-8182-55a5b410338d", "content-length": "702", "date": "Wed, 30 Apr 2025 06:44:21 GMT"} })
138092025-04-30T06:44:21.423ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1/component/rot/cfpa
138102025-04-30T06:44:21.424ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "d5ff2a93-6823-4a0b-94da-a050e65766ba", "content-length": "727", "date": "Wed, 30 Apr 2025 06:44:21 GMT"} })
138112025-04-30T06:44:21.425ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1/component/rot/cfpa
138122025-04-30T06:44:21.426ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "4b65bdaa-d35b-4bfa-af81-69a3a67fc944", "content-length": "729", "date": "Wed, 30 Apr 2025 06:44:21 GMT"} })
138132025-04-30T06:44:21.427ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1/component/rot/cfpa
138142025-04-30T06:44:21.427ZTRACtest_utilization (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory
138152025-04-30T06:44:21.427ZTRACtest_utilization (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
138162025-04-30T06:44:21.427ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
138172025-04-30T06:44:21.427ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
138182025-04-30T06:44:21.427ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
138192025-04-30T06:44:21.428ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "154e2aec-5953-4cb4-a484-6d01a3cb4af4", "content-length": "728", "date": "Wed, 30 Apr 2025 06:44:21 GMT"} })
138202025-04-30T06:44:21.429ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0
138212025-04-30T06:44:21.430ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "366dcb9c-2ddd-497c-8f84-b4c8ebf288f3", "content-length": "723", "date": "Wed, 30 Apr 2025 06:44:21 GMT"} })
138222025-04-30T06:44:21.431ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0/component/sp/caboose?firmware_slot=0
138232025-04-30T06:44:21.431ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
138242025-04-30T06:44:21.438ZDEBGtest_utilization (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory result = Ok(())
138252025-04-30T06:44:21.438ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
138262025-04-30T06:44:21.438ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
138272025-04-30T06:44:21.438ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
138282025-04-30T06:44:21.441ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
138292025-04-30T06:44:21.441ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
138302025-04-30T06:44:21.442ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
138312025-04-30T06:44:21.442ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "d0ddb131-7bce-4646-b4cc-b8487cd0e064", "content-length": "109", "date": "Wed, 30 Apr 2025 06:44:21 GMT"} })
138322025-04-30T06:44:21.442ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0/component/sp/caboose?firmware_slot=1
138332025-04-30T06:44:21.451ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "58760fa6-5818-49e0-bfd1-ac694b158d26", "content-length": "109", "date": "Wed, 30 Apr 2025 06:44:21 GMT"} })
138342025-04-30T06:44:21.451ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0/component/rot/caboose?firmware_slot=0
138352025-04-30T06:44:21.457ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "0ba68ab4-bcd0-4ff5-bd63-3a2455d4a3f4", "content-length": "169", "date": "Wed, 30 Apr 2025 06:44:21 GMT"} })
138362025-04-30T06:44:21.457ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0/component/rot/caboose?firmware_slot=1
138372025-04-30T06:44:21.458ZWARNtest_utilization (results-sink-cluster): failed to insert some results into metric DB collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 error = Failed to check out connection to database: No backends found for this service file = oximeter/collector/src/results_sink.rs:92
138382025-04-30T06:44:21.460ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "ebc57217-7039-46b8-b829-d3f1760db7b3", "content-length": "169", "date": "Wed, 30 Apr 2025 06:44:21 GMT"} })
138392025-04-30T06:44:21.460ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0/component/stage0/caboose?firmware_slot=0
138402025-04-30T06:44:21.463ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "6902d69b-6cf8-4310-8c31-3fe8394df069", "content-length": "178", "date": "Wed, 30 Apr 2025 06:44:21 GMT"} })
138412025-04-30T06:44:21.463ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0/component/stage0/caboose?firmware_slot=1
138422025-04-30T06:44:21.468ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "c543225a-a96e-4737-80be-7933b02d4caa", "content-length": "178", "date": "Wed, 30 Apr 2025 06:44:21 GMT"} })
138432025-04-30T06:44:21.468ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0/component/rot/cmpa
138442025-04-30T06:44:21.471ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "6b3c36b3-5790-4a42-8691-12e588df8d5b", "content-length": "702", "date": "Wed, 30 Apr 2025 06:44:21 GMT"} })
138452025-04-30T06:44:21.471ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0/component/rot/cfpa
138462025-04-30T06:44:21.473ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "663845f2-8a50-4c9a-b027-7b95f6810e92", "content-length": "727", "date": "Wed, 30 Apr 2025 06:44:21 GMT"} })
138472025-04-30T06:44:21.473ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0/component/rot/cfpa
138482025-04-30T06:44:21.475ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "29982058-4559-47f1-8d32-91adaaba4a90", "content-length": "729", "date": "Wed, 30 Apr 2025 06:44:21 GMT"} })
138492025-04-30T06:44:21.475ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0/component/rot/cfpa
138502025-04-30T06:44:21.477ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "4fcd9f03-c45b-4170-88e4-7037d5029905", "content-length": "728", "date": "Wed, 30 Apr 2025 06:44:21 GMT"} })
138512025-04-30T06:44:21.477ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1
138522025-04-30T06:44:21.479ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1", status: 200, headers: {"content-type": "application/json", "x-request-id": "90d7abe1-6243-4839-9d16-241b75629967", "content-length": "723", "date": "Wed, 30 Apr 2025 06:44:21 GMT"} })
138532025-04-30T06:44:21.480ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1/component/sp/caboose?firmware_slot=0
138542025-04-30T06:44:21.486ZDEBGtest_utilization: attempt to fetch latest inventory collection result = Ok("found")
138552025-04-30T06:44:21.488ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "a0b51de0-2a20-4125-91d1-712fb99c2e9a", "content-length": "109", "date": "Wed, 30 Apr 2025 06:44:21 GMT"} })
138562025-04-30T06:44:21.488ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1/component/sp/caboose?firmware_slot=1
138572025-04-30T06:44:21.493ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "ac2aa7d0-610a-4323-aa39-65770f8f16f9", "content-length": "109", "date": "Wed, 30 Apr 2025 06:44:21 GMT"} })
138582025-04-30T06:44:21.493ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1/component/rot/caboose?firmware_slot=0
138592025-04-30T06:44:21.501ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "faae7e0c-2b01-4012-a561-87b9d847dcb2", "content-length": "169", "date": "Wed, 30 Apr 2025 06:44:21 GMT"} })
138602025-04-30T06:44:21.502ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1/component/rot/caboose?firmware_slot=1
138612025-04-30T06:44:21.506ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "065d170a-1add-454c-ba14-96416d36dc84", "content-length": "169", "date": "Wed, 30 Apr 2025 06:44:21 GMT"} })
138622025-04-30T06:44:21.506ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1/component/stage0/caboose?firmware_slot=0
138632025-04-30T06:44:21.508ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "fd08fa57-1109-433a-a044-83592a641b28", "content-length": "178", "date": "Wed, 30 Apr 2025 06:44:21 GMT"} })
138642025-04-30T06:44:21.508ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1/component/stage0/caboose?firmware_slot=1
138652025-04-30T06:44:21.513ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "ed691fe4-3e08-4f9c-bc3f-eefdcb69935a", "content-length": "178", "date": "Wed, 30 Apr 2025 06:44:21 GMT"} })
138662025-04-30T06:44:21.514ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1/component/rot/cmpa
138672025-04-30T06:44:21.515ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "b826df9b-b7a2-4f4c-a283-9eb0e6d3440f", "content-length": "702", "date": "Wed, 30 Apr 2025 06:44:21 GMT"} })
138682025-04-30T06:44:21.516ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1/component/rot/cfpa
138692025-04-30T06:44:21.519ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "cc1be994-2424-4654-8887-5283e2bdab52", "content-length": "727", "date": "Wed, 30 Apr 2025 06:44:21 GMT"} })
138702025-04-30T06:44:21.520ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1/component/rot/cfpa
138712025-04-30T06:44:21.521ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "ab8eb6a7-34e9-49c6-9d52-548d9e3cf308", "content-length": "729", "date": "Wed, 30 Apr 2025 06:44:21 GMT"} })
138722025-04-30T06:44:21.521ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1/component/rot/cfpa
138732025-04-30T06:44:21.523ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "3b37cb7d-5eb5-41d9-9d9a-b4b6cc82ab5c", "content-length": "728", "date": "Wed, 30 Apr 2025 06:44:21 GMT"} })
138742025-04-30T06:44:21.523ZTRACtest_utilization (BackgroundTasks): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet
138752025-04-30T06:44:21.523ZTRACtest_utilization (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
138762025-04-30T06:44:21.524ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
138772025-04-30T06:44:21.524ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
138782025-04-30T06:44:21.526ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
138792025-04-30T06:44:21.541ZTRACtest_utilization (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory
138802025-04-30T06:44:21.547ZTRACtest_utilization (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
138812025-04-30T06:44:21.548ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
138822025-04-30T06:44:21.548ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
138832025-04-30T06:44:21.549ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
138842025-04-30T06:44:21.550ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
138852025-04-30T06:44:21.559ZDEBGtest_utilization (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet result = Ok(())
138862025-04-30T06:44:21.559ZTRACtest_utilization (BackgroundTasks): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet
138872025-04-30T06:44:21.559ZTRACtest_utilization (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
138882025-04-30T06:44:21.559ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
138892025-04-30T06:44:21.559ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
138902025-04-30T06:44:21.560ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
138912025-04-30T06:44:21.564ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
138922025-04-30T06:44:21.574ZDEBGtest_utilization (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory result = Ok(())
138932025-04-30T06:44:21.574ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
138942025-04-30T06:44:21.574ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
138952025-04-30T06:44:21.575ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
138962025-04-30T06:44:21.578ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
138972025-04-30T06:44:21.583ZDEBGtest_utilization (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet result = Ok(())
138982025-04-30T06:44:21.583ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
138992025-04-30T06:44:21.583ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
139002025-04-30T06:44:21.584ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
139012025-04-30T06:44:21.590ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
139022025-04-30T06:44:21.590ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
139032025-04-30T06:44:21.592ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
139042025-04-30T06:44:21.650ZDEBGtest_utilization (BackgroundTasks): begin collection from Sled Agent background_task = inventory_collection sled_agent_url = http://[::1]:39365
139052025-04-30T06:44:21.650ZDEBGtest_utilization (BackgroundTasks): client request SledAgent = http://[::1]:39365 background_task = inventory_collection body = None method = GET uri = http://[::1]:39365/inventory
139062025-04-30T06:44:21.652ZINFOtest_utilization (dropshot): accepted connection local_addr = [::1]:39365 remote_addr = [::1]:48029 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
139072025-04-30T06:44:21.662ZTRACtest_utilization (dropshot): incoming request local_addr = [::1]:39365 method = GET remote_addr = [::1]:48029 req_id = a6f3bc64-947a-4954-bae7-d30fa331a388 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /inventory
139082025-04-30T06:44:21.662ZINFOtest_utilization (dropshot): request completed latency_us = 579 local_addr = [::1]:39365 method = GET remote_addr = [::1]:48029 req_id = a6f3bc64-947a-4954-bae7-d30fa331a388 response_code = 200 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /inventory
139092025-04-30T06:44:21.666ZDEBGtest_utilization (BackgroundTasks): client response SledAgent = http://[::1]:39365 background_task = inventory_collection result = Ok(Response { url: "http://[::1]:39365/inventory", status: 200, headers: {"content-type": "application/json", "x-request-id": "a6f3bc64-947a-4954-bae7-d30fa331a388", "content-length": "3141", "date": "Wed, 30 Apr 2025 06:44:21 GMT"} })
139102025-04-30T06:44:21.669ZDEBGtest_utilization (BackgroundTasks): begin collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
139112025-04-30T06:44:21.669ZDEBGtest_utilization (BackgroundTasks): end collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
139122025-04-30T06:44:21.669ZDEBGtest_utilization (BackgroundTasks): finished collection background_task = inventory_collection
139132025-04-30T06:44:21.672ZTRACtest_utilization (BackgroundTasks): authorize begin action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory
139142025-04-30T06:44:21.672ZTRACtest_utilization (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
139152025-04-30T06:44:21.672ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
139162025-04-30T06:44:21.672ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
139172025-04-30T06:44:21.672ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
139182025-04-30T06:44:21.679ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
139192025-04-30T06:44:21.684ZDEBGtest_utilization (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory result = Ok(())
139202025-04-30T06:44:21.684ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
139212025-04-30T06:44:21.684ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
139222025-04-30T06:44:21.685ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
139232025-04-30T06:44:21.731ZDEBGtest_utilization: attempt to fetch latest inventory collection result = Ok("found")
139242025-04-30T06:44:21.732ZWARNtest_utilization (datastore): Non-retryable transaction failure err = DatabaseError(UniqueViolation, "duplicate key value violates unique constraint \\"inv_caboose_pkey\\"") transaction = inventory_insert_collection
139252025-04-30T06:44:21.732ZWARNtest_utilization (BackgroundTasks): inventory collection failed background_task = inventory_collection error = failed to collect inventory: saving inventory to database: Internal Error: unexpected database error: duplicate key value violates unique constraint "inv_caboose_pkey"
139262025-04-30T06:44:21.732ZDEBGtest_utilization (BackgroundTasks): activation complete background_task = inventory_collection elapsed = 1.304339972s iteration = 37 status = {"error":"failed to collect inventory: saving inventory to database: Internal Error: unexpected database error: duplicate key value violates unique constraint \\"inv_caboose_pkey\\""}
139272025-04-30T06:44:21.732ZDEBGtest_utilization (BackgroundTasks): activating background_task = inventory_collection iteration = 38 reason = Signaled
139282025-04-30T06:44:21.733ZTRACtest_utilization (BackgroundTasks): authorize begin action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory
139292025-04-30T06:44:21.733ZTRACtest_utilization (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
139302025-04-30T06:44:21.733ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
139312025-04-30T06:44:21.733ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
139322025-04-30T06:44:21.734ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
139332025-04-30T06:44:21.737ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
139342025-04-30T06:44:21.743ZDEBGtest_utilization (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory result = Ok(())
139352025-04-30T06:44:21.743ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
139362025-04-30T06:44:21.743ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
139372025-04-30T06:44:21.743ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
139382025-04-30T06:44:21.747ZDEBGtest_utilization (BackgroundTasks): inventory_prune_one: nothing eligible for removal (too few) background_task = inventory_collection candidates = [(13d8b8d9-6c6b-47a6-9190-952544edb75b, 0)]
139392025-04-30T06:44:21.747ZTRACtest_utilization (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal
139402025-04-30T06:44:21.748ZTRACtest_utilization (dns): buffer buffer = Length: 67 (0x43) bytes\n0000: d7 1e 01 00 00 01 00 00 00 00 00 01 04 5f 6d 67 ............._mg\n0010: 73 04 5f 74 63 70 0d 63 6f 6e 74 72 6f 6c 2d 70 s._tcp.control-p\n0020: 6c 61 6e 65 05 6f 78 69 64 65 08 69 6e 74 65 72 lane.oxide.inter\n0030: 6e 61 6c 00 00 21 00 01 00 00 29 04 d0 00 00 00 nal..!....).....\n0040: 00 00 00 ... kind = dns peer_addr = [::1]:54546 req_id = 817992d9-f152-46e8-9d08-1df34ade35ac
139412025-04-30T06:44:21.756ZDEBGtest_utilization (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 55070,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_mgs._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_mgs._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 4,\n 95,\n 109,\n 103,\n 115,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:54546 req_id = 817992d9-f152-46e8-9d08-1df34ade35ac
139422025-04-30T06:44:21.756ZDEBGtest_utilization (store): query key key = _mgs._tcp
139432025-04-30T06:44:21.757ZDEBGtest_utilization (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
139442025-04-30T06:44:21.757ZDEBGtest_utilization (dns): dns response additional_records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal"), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] kind = dns peer_addr = [::1]:54546 query = LowerQuery { name: LowerName(Name("_mgs._tcp.control-plane.oxide.internal.")), original: Query { name: Name("_mgs._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN } } records = [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 48759, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal") })) }] req_id = 817992d9-f152-46e8-9d08-1df34ade35ac
139452025-04-30T06:44:21.757ZDEBGtest_utilization (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgs._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 48759, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }], valid_until: Instant { tv_sec: 4326, tv_nsec: 556804041 } })
139462025-04-30T06:44:21.758ZTRACtest_utilization (DnsResolver): lookup_service_targets: looking up SRV target
139472025-04-30T06:44:21.758ZTRACtest_utilization (dns): buffer buffer = Length: 108 (0x6c) bytes\n0000: 56 a1 01 00 00 01 00 00 00 00 00 01 2d 64 65 6e V...........-den\n0010: 64 72 69 74 65 2d 62 36 64 36 35 33 34 31 2d 31 drite-b6d65341-1\n0020: 36 37 63 2d 34 31 64 66 2d 39 62 35 63 2d 34 31 67c-41df-9b5c-41\n0030: 63 64 65 64 39 39 63 32 32 39 04 68 6f 73 74 0d cded99c229.host.\n0040: 63 6f 6e 74 72 6f 6c 2d 70 6c 61 6e 65 05 6f 78 control-plane.ox\n0050: 69 64 65 08 69 6e 74 65 72 6e 61 6c 00 00 1c 00 ide.internal....\n0060: 01 00 00 29 04 d0 00 00 00 00 00 00 ...)........ kind = dns peer_addr = [::1]:49918 req_id = 03dd609d-2085-47d2-9bae-d78a82116594
139482025-04-30T06:44:21.758ZDEBGtest_utilization (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 22177,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n query_type: AAAA,\n query_class: IN,\n },\n },\n original: [\n 45,\n 100,\n 101,\n 110,\n 100,\n 114,\n 105,\n 116,\n 101,\n 45,\n 98,\n 54,\n 100,\n 54,\n 53,\n 51,\n 52,\n 49,\n 45,\n 49,\n 54,\n 55,\n 99,\n 45,\n 52,\n 49,\n 100,\n 102,\n 45,\n 57,\n 98,\n 53,\n 99,\n 45,\n 52,\n 49,\n 99,\n 100,\n 101,\n 100,\n 57,\n 57,\n 99,\n 50,\n 50,\n 57,\n 4,\n 104,\n 111,\n 115,\n 116,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 28,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:49918 req_id = 03dd609d-2085-47d2-9bae-d78a82116594
139492025-04-30T06:44:21.758ZDEBGtest_utilization (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
139502025-04-30T06:44:21.759ZDEBGtest_utilization (dns): dns response additional_records = [] kind = dns peer_addr = [::1]:49918 query = LowerQuery { name: LowerName(Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.")), original: Query { name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), query_type: AAAA, query_class: IN } } records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] req_id = 03dd609d-2085-47d2-9bae-d78a82116594
139512025-04-30T06:44:21.849ZTRACtest_utilization (DnsResolver): lookup_all_socket_v6 srv dns_name = _clickhouse-admin-keeper._tcp.control-plane.oxide.internal
139522025-04-30T06:44:21.858ZTRACtest_utilization (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory
139532025-04-30T06:44:21.858ZTRACtest_utilization (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
139542025-04-30T06:44:21.858ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
139552025-04-30T06:44:21.858ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
139562025-04-30T06:44:21.860ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
139572025-04-30T06:44:21.862ZTRACtest_utilization (dns): buffer buffer = Length: 87 (0x57) bytes\n0000: 58 3b 01 00 00 01 00 00 00 00 00 01 18 5f 63 6c X;..........._cl\n0010: 69 63 6b 68 6f 75 73 65 2d 61 64 6d 69 6e 2d 6b ickhouse-admin-k\n0020: 65 65 70 65 72 04 5f 74 63 70 0d 63 6f 6e 74 72 eeper._tcp.contr\n0030: 6f 6c 2d 70 6c 61 6e 65 05 6f 78 69 64 65 08 69 ol-plane.oxide.i\n0040: 6e 74 65 72 6e 61 6c 00 00 21 00 01 00 00 29 04 nternal..!....).\n0050: d0 00 00 00 00 00 00 ....... kind = dns peer_addr = [::1]:63157 req_id = a1c978f5-ad5a-4f1a-bec2-b6dae0f7da59
139582025-04-30T06:44:21.862ZDEBGtest_utilization (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 22587,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 24,\n 95,\n 99,\n 108,\n 105,\n 99,\n 107,\n 104,\n 111,\n 117,\n 115,\n 101,\n 45,\n 97,\n 100,\n 109,\n 105,\n 110,\n 45,\n 107,\n 101,\n 101,\n 112,\n 101,\n 114,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:63157 req_id = a1c978f5-ad5a-4f1a-bec2-b6dae0f7da59
139592025-04-30T06:44:21.863ZDEBGtest_utilization (store): query key key = _clickhouse-admin-keeper._tcp
139602025-04-30T06:44:21.863ZERROtest_utilization (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 22587, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 1, }, query: WireQuery { query: LowerQuery { name: LowerName( Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), ), original: Query { name: Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN, }, }, original: [ 24, 95, 99, 108, 105, 99, 107, 104, 111, 117, 115, 101, 45, 97, 100, 109, 105, 110, 45, 107, 101, 101, 112, 101, 114, 4, 95, 116, 99, 112, 13, 99, 111, 110, 116, 114, 111, 108, 45, 112, 108, 97, 110, 101, 5, 111, 120, 105, 100, 101, 8, 105, 110, 116, 101, 114, 110, 97, 108, 0, 0, 33, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: Some( Edns { rcode_high: 0, version: 0, dnssec_ok: false, max_payload: 1232, options: OPT { options: {}, }, }, ), } NXDOMAIN: no records found for name: "_clickhouse-admin-keeper._tcp.control-plane.oxide.internal." kind = dns peer_addr = [::1]:63157 req_id = a1c978f5-ad5a-4f1a-bec2-b6dae0f7da59
139612025-04-30T06:44:21.863ZDEBGtest_utilization (BackgroundTasks): begin collection background_task = inventory_collection
139622025-04-30T06:44:21.863ZDEBGtest_utilization (BackgroundTasks): begin collection from MGS background_task = inventory_collection mgs_url = http://[::1]:48759
139632025-04-30T06:44:21.863ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/ignition
139642025-04-30T06:44:21.865ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
139652025-04-30T06:44:21.873ZDEBGtest_utilization (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory result = Ok(())
139662025-04-30T06:44:21.873ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
139672025-04-30T06:44:21.873ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
139682025-04-30T06:44:21.873ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
139692025-04-30T06:44:21.876ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/ignition", status: 200, headers: {"content-type": "application/json", "x-request-id": "a57d04f1-27e4-4bab-b6eb-54cee5b28345", "content-length": "835", "date": "Wed, 30 Apr 2025 06:44:21 GMT"} })
139702025-04-30T06:44:21.876ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0
139712025-04-30T06:44:21.876ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
139722025-04-30T06:44:21.876ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
139732025-04-30T06:44:21.877ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
139742025-04-30T06:44:21.883ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "450e845a-b84f-42c7-8042-5e695d2f84fa", "content-length": "734", "date": "Wed, 30 Apr 2025 06:44:21 GMT"} })
139752025-04-30T06:44:21.887ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0/component/sp/caboose?firmware_slot=0
139762025-04-30T06:44:21.891ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "25e7c848-e3ca-4609-8ba8-45658fed1e65", "content-length": "111", "date": "Wed, 30 Apr 2025 06:44:21 GMT"} })
139772025-04-30T06:44:21.891ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0/component/sp/caboose?firmware_slot=1
139782025-04-30T06:44:21.896ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "8b22cde3-3923-4c2d-8c82-6d7e828cd458", "content-length": "111", "date": "Wed, 30 Apr 2025 06:44:21 GMT"} })
139792025-04-30T06:44:21.896ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0/component/rot/caboose?firmware_slot=0
139802025-04-30T06:44:21.899ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "6c438887-2d66-4c1e-b5e4-13335ecd731c", "content-length": "170", "date": "Wed, 30 Apr 2025 06:44:21 GMT"} })
139812025-04-30T06:44:21.899ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0/component/rot/caboose?firmware_slot=1
139822025-04-30T06:44:21.902ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "3dde30e3-685c-4127-a6d6-6e8f09de5894", "content-length": "170", "date": "Wed, 30 Apr 2025 06:44:21 GMT"} })
139832025-04-30T06:44:21.902ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0/component/stage0/caboose?firmware_slot=0
139842025-04-30T06:44:21.905ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "60c99135-f9f8-4a5d-a408-0f2b15cf218b", "content-length": "179", "date": "Wed, 30 Apr 2025 06:44:21 GMT"} })
139852025-04-30T06:44:21.905ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0/component/stage0/caboose?firmware_slot=1
139862025-04-30T06:44:21.907ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "1ce81e7a-0a35-4a21-bb60-2a44ab83414c", "content-length": "179", "date": "Wed, 30 Apr 2025 06:44:21 GMT"} })
139872025-04-30T06:44:21.908ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0/component/rot/cmpa
139882025-04-30T06:44:21.910ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "5180ce81-4ebf-4555-8031-d1fe14eb4449", "content-length": "702", "date": "Wed, 30 Apr 2025 06:44:21 GMT"} })
139892025-04-30T06:44:21.911ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0/component/rot/cfpa
139902025-04-30T06:44:21.912ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "317482c5-6df4-4c09-90da-dd4bb4376e1e", "content-length": "727", "date": "Wed, 30 Apr 2025 06:44:21 GMT"} })
139912025-04-30T06:44:21.912ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0/component/rot/cfpa
139922025-04-30T06:44:21.916ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "1fbdc352-a0c5-47cf-990a-ce16d6e9ca35", "content-length": "729", "date": "Wed, 30 Apr 2025 06:44:21 GMT"} })
139932025-04-30T06:44:21.916ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0/component/rot/cfpa
139942025-04-30T06:44:21.919ZDEBGtest_utilization: attempt to fetch latest inventory collection result = Ok("found")
139952025-04-30T06:44:21.920ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "92b29b17-59ab-4710-bb15-fb1b9c0a4bf8", "content-length": "728", "date": "Wed, 30 Apr 2025 06:44:21 GMT"} })
139962025-04-30T06:44:21.920ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1
139972025-04-30T06:44:21.922ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1", status: 200, headers: {"content-type": "application/json", "x-request-id": "f090f662-5d2d-484a-a2da-965310120d72", "content-length": "734", "date": "Wed, 30 Apr 2025 06:44:21 GMT"} })
139982025-04-30T06:44:21.922ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1/component/sp/caboose?firmware_slot=0
139992025-04-30T06:44:21.924ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "095257a7-10d4-4ed3-a58e-ccf7fcefbd03", "content-length": "111", "date": "Wed, 30 Apr 2025 06:44:21 GMT"} })
140002025-04-30T06:44:21.924ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1/component/sp/caboose?firmware_slot=1
140012025-04-30T06:44:21.925ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "449ece37-2e73-456e-b72b-d8d13b971687", "content-length": "111", "date": "Wed, 30 Apr 2025 06:44:21 GMT"} })
140022025-04-30T06:44:21.926ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1/component/rot/caboose?firmware_slot=0
140032025-04-30T06:44:21.927ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "c83080a5-d8d7-4445-99a2-b2d5b5a26cfa", "content-length": "170", "date": "Wed, 30 Apr 2025 06:44:21 GMT"} })
140042025-04-30T06:44:21.927ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1/component/rot/caboose?firmware_slot=1
140052025-04-30T06:44:21.930ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "aaa87531-26c2-4017-bc9c-56855b2332b3", "content-length": "170", "date": "Wed, 30 Apr 2025 06:44:21 GMT"} })
140062025-04-30T06:44:21.930ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1/component/stage0/caboose?firmware_slot=0
140072025-04-30T06:44:21.933ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "a127d1c9-5348-4a27-8987-c6ae3aa16bdf", "content-length": "179", "date": "Wed, 30 Apr 2025 06:44:21 GMT"} })
140082025-04-30T06:44:21.933ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1/component/stage0/caboose?firmware_slot=1
140092025-04-30T06:44:21.934ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "947ce883-eee6-4dfc-9f72-edab7ac05203", "content-length": "179", "date": "Wed, 30 Apr 2025 06:44:21 GMT"} })
140102025-04-30T06:44:21.935ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1/component/rot/cmpa
140112025-04-30T06:44:21.936ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "64bf1934-3173-40d5-ada3-7fb63113975d", "content-length": "702", "date": "Wed, 30 Apr 2025 06:44:21 GMT"} })
140122025-04-30T06:44:21.936ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1/component/rot/cfpa
140132025-04-30T06:44:21.937ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "6668cb6d-1eed-45c5-bf48-06d99207d639", "content-length": "727", "date": "Wed, 30 Apr 2025 06:44:21 GMT"} })
140142025-04-30T06:44:21.937ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1/component/rot/cfpa
140152025-04-30T06:44:21.938ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "86319086-55a0-41f9-9028-38427831f7e2", "content-length": "729", "date": "Wed, 30 Apr 2025 06:44:21 GMT"} })
140162025-04-30T06:44:21.938ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1/component/rot/cfpa
140172025-04-30T06:44:21.940ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "61f5d55f-d750-41da-a587-fe59f26ce8d1", "content-length": "728", "date": "Wed, 30 Apr 2025 06:44:21 GMT"} })
140182025-04-30T06:44:21.940ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0
140192025-04-30T06:44:21.941ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "cdecd824-16b7-4e6d-96dc-a65e3df63307", "content-length": "723", "date": "Wed, 30 Apr 2025 06:44:21 GMT"} })
140202025-04-30T06:44:21.941ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0/component/sp/caboose?firmware_slot=0
140212025-04-30T06:44:21.944ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "89fdd9d1-8591-413a-8343-ba1892d31b2b", "content-length": "109", "date": "Wed, 30 Apr 2025 06:44:21 GMT"} })
140222025-04-30T06:44:21.944ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0/component/sp/caboose?firmware_slot=1
140232025-04-30T06:44:21.947ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "ab7cbecb-2792-490b-a7e7-f81a79e1b2e4", "content-length": "109", "date": "Wed, 30 Apr 2025 06:44:21 GMT"} })
140242025-04-30T06:44:21.947ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0/component/rot/caboose?firmware_slot=0
140252025-04-30T06:44:21.949ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "43b6f5cb-7a1f-484a-9188-7276adc8f599", "content-length": "169", "date": "Wed, 30 Apr 2025 06:44:21 GMT"} })
140262025-04-30T06:44:21.949ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0/component/rot/caboose?firmware_slot=1
140272025-04-30T06:44:21.951ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "3730bf74-7a49-4f28-b31e-fe1b44545b96", "content-length": "169", "date": "Wed, 30 Apr 2025 06:44:21 GMT"} })
140282025-04-30T06:44:21.951ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0/component/stage0/caboose?firmware_slot=0
140292025-04-30T06:44:21.954ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "4128e24a-d486-46f3-b884-d56f07c4d5f0", "content-length": "178", "date": "Wed, 30 Apr 2025 06:44:21 GMT"} })
140302025-04-30T06:44:21.954ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0/component/stage0/caboose?firmware_slot=1
140312025-04-30T06:44:21.958ZINFOtest_utilization (oximeter-agent): refreshing list of producers from Nexus collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 file = oximeter/collector/src/agent.rs:562
140322025-04-30T06:44:21.959ZDEBGtest_utilization (oximeter): client request body = None method = GET uri = http://127.0.0.1:32956/metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?sort_by=id_ascending
140332025-04-30T06:44:21.959ZTRACtest_utilization (dropshot_internal): incoming request local_addr = 127.0.0.1:32956 method = GET remote_addr = 127.0.0.1:55797 req_id = e0740bac-cffa-4465-8f58-57e89fa0a68c uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?sort_by=id_ascending
140342025-04-30T06:44:21.959ZTRACtest_utilization (dropshot_internal): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = 127.0.0.1:32956 method = GET remote_addr = 127.0.0.1:55797 req_id = e0740bac-cffa-4465-8f58-57e89fa0a68c resource = Database uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?sort_by=id_ascending
140352025-04-30T06:44:21.959ZDEBGtest_utilization (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = 127.0.0.1:32956 method = GET remote_addr = 127.0.0.1:55797 req_id = e0740bac-cffa-4465-8f58-57e89fa0a68c roles = RoleSet { roles: {} } uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?sort_by=id_ascending
140362025-04-30T06:44:21.963ZDEBGtest_utilization (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = 127.0.0.1:32956 method = GET remote_addr = 127.0.0.1:55797 req_id = e0740bac-cffa-4465-8f58-57e89fa0a68c resource = Database result = Ok(()) uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?sort_by=id_ascending
140372025-04-30T06:44:21.964ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "d2043624-130b-4acc-b788-fcc6bc921fd2", "content-length": "178", "date": "Wed, 30 Apr 2025 06:44:21 GMT"} })
140382025-04-30T06:44:21.964ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0/component/rot/cmpa
140392025-04-30T06:44:21.964ZINFOtest_utilization (dropshot_internal): request completed latency_us = 5433 local_addr = 127.0.0.1:32956 method = GET remote_addr = 127.0.0.1:55797 req_id = e0740bac-cffa-4465-8f58-57e89fa0a68c response_code = 200 uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?sort_by=id_ascending
140402025-04-30T06:44:21.966ZDEBGtest_utilization (oximeter): client response result = Ok(Response { url: "http://127.0.0.1:32956/metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?sort_by=id_ascending", status: 200, headers: {"content-type": "application/json", "x-request-id": "e0740bac-cffa-4465-8f58-57e89fa0a68c", "content-length": "405", "date": "Wed, 30 Apr 2025 06:44:21 GMT"} })
140412025-04-30T06:44:21.968ZDEBGtest_utilization (oximeter): client request body = None method = GET uri = http://127.0.0.1:32956/metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJhYzU1ZDhkOS1hMjk4LTQ1MzQtODk4Yi1iZWVlMWM5ZTRlNTIifX0%3D
140422025-04-30T06:44:21.973ZTRACtest_utilization (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory
140432025-04-30T06:44:21.973ZTRACtest_utilization (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
140442025-04-30T06:44:21.973ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
140452025-04-30T06:44:21.973ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
140462025-04-30T06:44:21.974ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
140472025-04-30T06:44:21.974ZTRACtest_utilization (dropshot_internal): incoming request local_addr = 127.0.0.1:32956 method = GET remote_addr = 127.0.0.1:55797 req_id = fe11fa5e-74da-4aa7-a7ad-2305d02f2318 uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJhYzU1ZDhkOS1hMjk4LTQ1MzQtODk4Yi1iZWVlMWM5ZTRlNTIifX0%3D
140482025-04-30T06:44:21.975ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "8f43ce71-92c1-487e-b09b-dccd0fb19e49", "content-length": "702", "date": "Wed, 30 Apr 2025 06:44:21 GMT"} })
140492025-04-30T06:44:21.975ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0/component/rot/cfpa
140502025-04-30T06:44:21.987ZTRACtest_utilization (dropshot_internal): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = 127.0.0.1:32956 method = GET remote_addr = 127.0.0.1:55797 req_id = fe11fa5e-74da-4aa7-a7ad-2305d02f2318 resource = Database uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJhYzU1ZDhkOS1hMjk4LTQ1MzQtODk4Yi1iZWVlMWM5ZTRlNTIifX0%3D
140512025-04-30T06:44:21.987ZDEBGtest_utilization (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = 127.0.0.1:32956 method = GET remote_addr = 127.0.0.1:55797 req_id = fe11fa5e-74da-4aa7-a7ad-2305d02f2318 roles = RoleSet { roles: {} } uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJhYzU1ZDhkOS1hMjk4LTQ1MzQtODk4Yi1iZWVlMWM5ZTRlNTIifX0%3D
140522025-04-30T06:44:21.987ZDEBGtest_utilization (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = 127.0.0.1:32956 method = GET remote_addr = 127.0.0.1:55797 req_id = fe11fa5e-74da-4aa7-a7ad-2305d02f2318 resource = Database result = Ok(()) uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJhYzU1ZDhkOS1hMjk4LTQ1MzQtODk4Yi1iZWVlMWM5ZTRlNTIifX0%3D
140532025-04-30T06:44:21.988ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "033bde4f-a164-4f7b-8c66-e9e52d8f344a", "content-length": "727", "date": "Wed, 30 Apr 2025 06:44:21 GMT"} })
140542025-04-30T06:44:21.988ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0/component/rot/cfpa
140552025-04-30T06:44:21.990ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "5b9ffd67-f12e-4510-be5b-a9dfa74bb6f4", "content-length": "729", "date": "Wed, 30 Apr 2025 06:44:21 GMT"} })
140562025-04-30T06:44:21.990ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0/component/rot/cfpa
140572025-04-30T06:44:21.990ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
140582025-04-30T06:44:21.996ZDEBGtest_utilization (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory result = Ok(())
140592025-04-30T06:44:21.997ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
140602025-04-30T06:44:21.997ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
140612025-04-30T06:44:21.997ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
140622025-04-30T06:44:21.998ZINFOtest_utilization (dropshot_internal): request completed latency_us = 23548 local_addr = 127.0.0.1:32956 method = GET remote_addr = 127.0.0.1:55797 req_id = fe11fa5e-74da-4aa7-a7ad-2305d02f2318 response_code = 200 uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJhYzU1ZDhkOS1hMjk4LTQ1MzQtODk4Yi1iZWVlMWM5ZTRlNTIifX0%3D
140632025-04-30T06:44:21.998ZDEBGtest_utilization (collection-task): sent timer-based collection request to the collection task collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
140642025-04-30T06:44:21.998ZDEBGtest_utilization (oximeter): client response result = Ok(Response { url: "http://127.0.0.1:32956/metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJhYzU1ZDhkOS1hMjk4LTQ1MzQtODk4Yi1iZWVlMWM5ZTRlNTIifX0%3D", status: 200, headers: {"content-type": "application/json", "x-request-id": "fe11fa5e-74da-4aa7-a7ad-2305d02f2318", "content-length": "29", "date": "Wed, 30 Apr 2025 06:44:21 GMT"} })
140652025-04-30T06:44:21.999ZTRACtest_utilization (oximeter-agent): ignoring request to update existing metric producer, since the endpoint information is the same as the existing collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
140662025-04-30T06:44:21.999ZTRACtest_utilization (oximeter-agent): ignoring request to update existing metric producer, since the endpoint information is the same as the existing collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = ac55d8d9-a298-4534-898b-beee1c9e4e52
140672025-04-30T06:44:21.999ZINFOtest_utilization (oximeter-agent): refreshed list of producers from Nexus collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 file = oximeter/collector/src/agent.rs:637 n_current_tasks = 2 n_pruned_tasks = 0
140682025-04-30T06:44:21.999ZDEBGtest_utilization (collection-task): collecting from producer collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
140692025-04-30T06:44:22.001ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
140702025-04-30T06:44:22.002ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
140712025-04-30T06:44:22.003ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
140722025-04-30T06:44:22.004ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "51388f4e-dd4e-400e-8c1b-089cf7023e8a", "content-length": "728", "date": "Wed, 30 Apr 2025 06:44:21 GMT"} })
140732025-04-30T06:44:22.004ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1
140742025-04-30T06:44:22.004ZTRACtest_utilization (collection-task): sent collection request to producer collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
140752025-04-30T06:44:22.005ZDEBGtest_utilization (collection-task): collected results from producer collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 n_results = 1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
140762025-04-30T06:44:22.005ZTRACtest_utilization (collection-task): forwarded results to main collection loop collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
140772025-04-30T06:44:22.005ZTRACtest_utilization (collection-task): top of inner collection loop, waiting for next request collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
140782025-04-30T06:44:22.008ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1", status: 200, headers: {"content-type": "application/json", "x-request-id": "1873b9bd-5a1f-41a6-b260-00f896b32265", "content-length": "723", "date": "Wed, 30 Apr 2025 06:44:21 GMT"} })
140792025-04-30T06:44:22.008ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1/component/sp/caboose?firmware_slot=0
140802025-04-30T06:44:22.011ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "f6816e44-d7df-447c-8077-d5967ab7b83e", "content-length": "109", "date": "Wed, 30 Apr 2025 06:44:21 GMT"} })
140812025-04-30T06:44:22.011ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1/component/sp/caboose?firmware_slot=1
140822025-04-30T06:44:22.016ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "469e123e-71e6-4633-95db-b7857cfa515a", "content-length": "109", "date": "Wed, 30 Apr 2025 06:44:21 GMT"} })
140832025-04-30T06:44:22.017ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1/component/rot/caboose?firmware_slot=0
140842025-04-30T06:44:22.021ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "5e7e86e0-d770-4892-8f73-8a0dc2165499", "content-length": "169", "date": "Wed, 30 Apr 2025 06:44:21 GMT"} })
140852025-04-30T06:44:22.021ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1/component/rot/caboose?firmware_slot=1
140862025-04-30T06:44:22.023ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "79c8be0e-0cee-46b5-b8bf-b4a140922b06", "content-length": "169", "date": "Wed, 30 Apr 2025 06:44:21 GMT"} })
140872025-04-30T06:44:22.023ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1/component/stage0/caboose?firmware_slot=0
140882025-04-30T06:44:22.026ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "85f3bfb5-9a72-4151-a0de-c32d2b652883", "content-length": "178", "date": "Wed, 30 Apr 2025 06:44:21 GMT"} })
140892025-04-30T06:44:22.026ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1/component/stage0/caboose?firmware_slot=1
140902025-04-30T06:44:22.030ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "afccd585-7d4e-4528-bcce-d1185eab108a", "content-length": "178", "date": "Wed, 30 Apr 2025 06:44:21 GMT"} })
140912025-04-30T06:44:22.031ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1/component/rot/cmpa
140922025-04-30T06:44:22.035ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "fa57d3d3-a1e0-4bf0-9bf1-695b64d7264f", "content-length": "702", "date": "Wed, 30 Apr 2025 06:44:21 GMT"} })
140932025-04-30T06:44:22.035ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1/component/rot/cfpa
140942025-04-30T06:44:22.037ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "6dd30e99-5dba-441b-ba45-f936cbf62ca8", "content-length": "727", "date": "Wed, 30 Apr 2025 06:44:21 GMT"} })
140952025-04-30T06:44:22.037ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1/component/rot/cfpa
140962025-04-30T06:44:22.043ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "cfd32233-8dd0-4c8f-a113-06aca415f66d", "content-length": "729", "date": "Wed, 30 Apr 2025 06:44:21 GMT"} })
140972025-04-30T06:44:22.043ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1/component/rot/cfpa
140982025-04-30T06:44:22.047ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "7bfc4d1b-3e1f-464a-ae59-4e458bdaab4f", "content-length": "728", "date": "Wed, 30 Apr 2025 06:44:21 GMT"} })
140992025-04-30T06:44:22.047ZTRACtest_utilization (BackgroundTasks): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet
141002025-04-30T06:44:22.047ZTRACtest_utilization (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
141012025-04-30T06:44:22.047ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
141022025-04-30T06:44:22.047ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
141032025-04-30T06:44:22.049ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
141042025-04-30T06:44:22.049ZDEBGtest_utilization: attempt to fetch latest inventory collection result = Ok("found")
141052025-04-30T06:44:22.052ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
141062025-04-30T06:44:22.060ZDEBGtest_utilization (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet result = Ok(())
141072025-04-30T06:44:22.060ZTRACtest_utilization (BackgroundTasks): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet
141082025-04-30T06:44:22.060ZTRACtest_utilization (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
141092025-04-30T06:44:22.060ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
141102025-04-30T06:44:22.060ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
141112025-04-30T06:44:22.061ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
141122025-04-30T06:44:22.066ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
141132025-04-30T06:44:22.071ZDEBGtest_utilization (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet result = Ok(())
141142025-04-30T06:44:22.071ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
141152025-04-30T06:44:22.071ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
141162025-04-30T06:44:22.072ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
141172025-04-30T06:44:22.163ZDEBGtest_utilization (BackgroundTasks): begin collection from Sled Agent background_task = inventory_collection sled_agent_url = http://[::1]:39365
141182025-04-30T06:44:22.163ZDEBGtest_utilization (BackgroundTasks): client request SledAgent = http://[::1]:39365 background_task = inventory_collection body = None method = GET uri = http://[::1]:39365/inventory
141192025-04-30T06:44:22.163ZTRACtest_utilization (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory
141202025-04-30T06:44:22.163ZTRACtest_utilization (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
141212025-04-30T06:44:22.163ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
141222025-04-30T06:44:22.163ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
141232025-04-30T06:44:22.164ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
141242025-04-30T06:44:22.164ZINFOtest_utilization (dropshot): accepted connection local_addr = [::1]:39365 remote_addr = [::1]:33229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
141252025-04-30T06:44:22.165ZTRACtest_utilization (dropshot): incoming request local_addr = [::1]:39365 method = GET remote_addr = [::1]:33229 req_id = d99af95a-54d5-43fe-9591-6c6d34b87ab2 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /inventory
141262025-04-30T06:44:22.166ZINFOtest_utilization (dropshot): request completed latency_us = 1479 local_addr = [::1]:39365 method = GET remote_addr = [::1]:33229 req_id = d99af95a-54d5-43fe-9591-6c6d34b87ab2 response_code = 200 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /inventory
141272025-04-30T06:44:22.167ZDEBGtest_utilization (BackgroundTasks): client response SledAgent = http://[::1]:39365 background_task = inventory_collection result = Ok(Response { url: "http://[::1]:39365/inventory", status: 200, headers: {"content-type": "application/json", "x-request-id": "d99af95a-54d5-43fe-9591-6c6d34b87ab2", "content-length": "3141", "date": "Wed, 30 Apr 2025 06:44:22 GMT"} })
141282025-04-30T06:44:22.177ZDEBGtest_utilization (BackgroundTasks): begin collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
141292025-04-30T06:44:22.177ZDEBGtest_utilization (BackgroundTasks): end collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
141302025-04-30T06:44:22.177ZDEBGtest_utilization (BackgroundTasks): finished collection background_task = inventory_collection
141312025-04-30T06:44:22.183ZTRACtest_utilization (BackgroundTasks): authorize begin action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory
141322025-04-30T06:44:22.183ZTRACtest_utilization (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
141332025-04-30T06:44:22.183ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
141342025-04-30T06:44:22.183ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
141352025-04-30T06:44:22.184ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
141362025-04-30T06:44:22.184ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
141372025-04-30T06:44:22.190ZDEBGtest_utilization (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory result = Ok(())
141382025-04-30T06:44:22.190ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
141392025-04-30T06:44:22.190ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
141402025-04-30T06:44:22.191ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
141412025-04-30T06:44:22.194ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
141422025-04-30T06:44:22.198ZDEBGtest_utilization (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory result = Ok(())
141432025-04-30T06:44:22.198ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
141442025-04-30T06:44:22.198ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
141452025-04-30T06:44:22.198ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
141462025-04-30T06:44:22.202ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
141472025-04-30T06:44:22.202ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
141482025-04-30T06:44:22.204ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
141492025-04-30T06:44:22.238ZWARNtest_utilization (datastore): Non-retryable transaction failure err = DatabaseError(UniqueViolation, "duplicate key value violates unique constraint \\"inv_caboose_pkey\\"") transaction = inventory_insert_collection
141502025-04-30T06:44:22.238ZWARNtest_utilization (BackgroundTasks): inventory collection failed background_task = inventory_collection error = failed to collect inventory: saving inventory to database: Internal Error: unexpected database error: duplicate key value violates unique constraint "inv_caboose_pkey"
141512025-04-30T06:44:22.238ZDEBGtest_utilization (BackgroundTasks): activation complete background_task = inventory_collection elapsed = 505.926716ms iteration = 38 status = {"error":"failed to collect inventory: saving inventory to database: Internal Error: unexpected database error: duplicate key value violates unique constraint \\"inv_caboose_pkey\\""}
141522025-04-30T06:44:22.238ZDEBGtest_utilization (BackgroundTasks): activating background_task = inventory_collection iteration = 39 reason = Signaled
141532025-04-30T06:44:22.238ZTRACtest_utilization (BackgroundTasks): authorize begin action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory
141542025-04-30T06:44:22.238ZTRACtest_utilization (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
141552025-04-30T06:44:22.239ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
141562025-04-30T06:44:22.239ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
141572025-04-30T06:44:22.240ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
141582025-04-30T06:44:22.246ZDEBGtest_utilization: attempt to fetch latest inventory collection result = Ok("found")
141592025-04-30T06:44:22.247ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
141602025-04-30T06:44:22.253ZDEBGtest_utilization (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory result = Ok(())
141612025-04-30T06:44:22.253ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
141622025-04-30T06:44:22.253ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
141632025-04-30T06:44:22.254ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
141642025-04-30T06:44:22.257ZDEBGtest_utilization (BackgroundTasks): inventory_prune_one: nothing eligible for removal (too few) background_task = inventory_collection candidates = [(13d8b8d9-6c6b-47a6-9190-952544edb75b, 0)]
141652025-04-30T06:44:22.257ZTRACtest_utilization (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal
141662025-04-30T06:44:22.258ZTRACtest_utilization (dns): buffer buffer = Length: 67 (0x43) bytes\n0000: d1 7d 01 00 00 01 00 00 00 00 00 01 04 5f 6d 67 .}..........._mg\n0010: 73 04 5f 74 63 70 0d 63 6f 6e 74 72 6f 6c 2d 70 s._tcp.control-p\n0020: 6c 61 6e 65 05 6f 78 69 64 65 08 69 6e 74 65 72 lane.oxide.inter\n0030: 6e 61 6c 00 00 21 00 01 00 00 29 04 d0 00 00 00 nal..!....).....\n0040: 00 00 00 ... kind = dns peer_addr = [::1]:57484 req_id = c973092d-f82f-4155-bfd7-d80f96bbd2b5
141672025-04-30T06:44:22.258ZDEBGtest_utilization (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 53629,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_mgs._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_mgs._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 4,\n 95,\n 109,\n 103,\n 115,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:57484 req_id = c973092d-f82f-4155-bfd7-d80f96bbd2b5
141682025-04-30T06:44:22.258ZDEBGtest_utilization (store): query key key = _mgs._tcp
141692025-04-30T06:44:22.258ZDEBGtest_utilization (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
141702025-04-30T06:44:22.258ZDEBGtest_utilization (dns): dns response additional_records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal"), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] kind = dns peer_addr = [::1]:57484 query = LowerQuery { name: LowerName(Name("_mgs._tcp.control-plane.oxide.internal.")), original: Query { name: Name("_mgs._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN } } records = [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 48759, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal") })) }] req_id = c973092d-f82f-4155-bfd7-d80f96bbd2b5
141712025-04-30T06:44:22.258ZDEBGtest_utilization (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgs._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 48759, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }], valid_until: Instant { tv_sec: 4327, tv_nsec: 57863138 } })
141722025-04-30T06:44:22.259ZTRACtest_utilization (DnsResolver): lookup_service_targets: looking up SRV target
141732025-04-30T06:44:22.259ZTRACtest_utilization (dns): buffer buffer = Length: 108 (0x6c) bytes\n0000: 9c 58 01 00 00 01 00 00 00 00 00 01 2d 64 65 6e .X..........-den\n0010: 64 72 69 74 65 2d 62 36 64 36 35 33 34 31 2d 31 drite-b6d65341-1\n0020: 36 37 63 2d 34 31 64 66 2d 39 62 35 63 2d 34 31 67c-41df-9b5c-41\n0030: 63 64 65 64 39 39 63 32 32 39 04 68 6f 73 74 0d cded99c229.host.\n0040: 63 6f 6e 74 72 6f 6c 2d 70 6c 61 6e 65 05 6f 78 control-plane.ox\n0050: 69 64 65 08 69 6e 74 65 72 6e 61 6c 00 00 1c 00 ide.internal....\n0060: 01 00 00 29 04 d0 00 00 00 00 00 00 ...)........ kind = dns peer_addr = [::1]:49940 req_id = e5facaf6-41b5-4c10-b1ba-629524f7a558
141742025-04-30T06:44:22.259ZDEBGtest_utilization (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 40024,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n query_type: AAAA,\n query_class: IN,\n },\n },\n original: [\n 45,\n 100,\n 101,\n 110,\n 100,\n 114,\n 105,\n 116,\n 101,\n 45,\n 98,\n 54,\n 100,\n 54,\n 53,\n 51,\n 52,\n 49,\n 45,\n 49,\n 54,\n 55,\n 99,\n 45,\n 52,\n 49,\n 100,\n 102,\n 45,\n 57,\n 98,\n 53,\n 99,\n 45,\n 52,\n 49,\n 99,\n 100,\n 101,\n 100,\n 57,\n 57,\n 99,\n 50,\n 50,\n 57,\n 4,\n 104,\n 111,\n 115,\n 116,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 28,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:49940 req_id = e5facaf6-41b5-4c10-b1ba-629524f7a558
141752025-04-30T06:44:22.259ZDEBGtest_utilization (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
141762025-04-30T06:44:22.259ZDEBGtest_utilization (dns): dns response additional_records = [] kind = dns peer_addr = [::1]:49940 query = LowerQuery { name: LowerName(Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.")), original: Query { name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), query_type: AAAA, query_class: IN } } records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] req_id = e5facaf6-41b5-4c10-b1ba-629524f7a558
141772025-04-30T06:44:22.387ZTRACtest_utilization (DnsResolver): lookup_all_socket_v6 srv dns_name = _clickhouse-admin-keeper._tcp.control-plane.oxide.internal
141782025-04-30T06:44:22.388ZTRACtest_utilization (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory
141792025-04-30T06:44:22.388ZTRACtest_utilization (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
141802025-04-30T06:44:22.388ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
141812025-04-30T06:44:22.388ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
141822025-04-30T06:44:22.389ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
141832025-04-30T06:44:22.390ZDEBGtest_utilization (results-sink): inserting 1 samples into database collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
141842025-04-30T06:44:22.390ZDEBGtest_utilization (results-sink-cluster): inserting 1 samples into database collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
141852025-04-30T06:44:22.391ZDEBGtest_utilization (clickhouse-client): unrolling 1 total samples collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 4d1485f0-110e-4161-9a33-c4610d68be5d
141862025-04-30T06:44:22.392ZTRACtest_utilization (clickhouse-client): inserting data collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 4d1485f0-110e-4161-9a33-c4610d68be5d n_columns = 4 n_rows = 2 sql = INSERT INTO oximeter.fields_string FORMAT Native
141872025-04-30T06:44:22.401ZTRACtest_utilization (dns): buffer buffer = Length: 87 (0x57) bytes\n0000: 3a e2 01 00 00 01 00 00 00 00 00 01 18 5f 63 6c :............_cl\n0010: 69 63 6b 68 6f 75 73 65 2d 61 64 6d 69 6e 2d 6b ickhouse-admin-k\n0020: 65 65 70 65 72 04 5f 74 63 70 0d 63 6f 6e 74 72 eeper._tcp.contr\n0030: 6f 6c 2d 70 6c 61 6e 65 05 6f 78 69 64 65 08 69 ol-plane.oxide.i\n0040: 6e 74 65 72 6e 61 6c 00 00 21 00 01 00 00 29 04 nternal..!....).\n0050: d0 00 00 00 00 00 00 ....... kind = dns peer_addr = [::1]:51301 req_id = a5f113e9-e9ae-4142-8d77-7cd2e75be74f
141882025-04-30T06:44:22.401ZDEBGtest_utilization (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 15074,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 24,\n 95,\n 99,\n 108,\n 105,\n 99,\n 107,\n 104,\n 111,\n 117,\n 115,\n 101,\n 45,\n 97,\n 100,\n 109,\n 105,\n 110,\n 45,\n 107,\n 101,\n 101,\n 112,\n 101,\n 114,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:51301 req_id = a5f113e9-e9ae-4142-8d77-7cd2e75be74f
141892025-04-30T06:44:22.401ZDEBGtest_utilization (store): query key key = _clickhouse-admin-keeper._tcp
141902025-04-30T06:44:22.401ZERROtest_utilization (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 15074, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 1, }, query: WireQuery { query: LowerQuery { name: LowerName( Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), ), original: Query { name: Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN, }, }, original: [ 24, 95, 99, 108, 105, 99, 107, 104, 111, 117, 115, 101, 45, 97, 100, 109, 105, 110, 45, 107, 101, 101, 112, 101, 114, 4, 95, 116, 99, 112, 13, 99, 111, 110, 116, 114, 111, 108, 45, 112, 108, 97, 110, 101, 5, 111, 120, 105, 100, 101, 8, 105, 110, 116, 101, 114, 110, 97, 108, 0, 0, 33, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: Some( Edns { rcode_high: 0, version: 0, dnssec_ok: false, max_payload: 1232, options: OPT { options: {}, }, }, ), } NXDOMAIN: no records found for name: "_clickhouse-admin-keeper._tcp.control-plane.oxide.internal." kind = dns peer_addr = [::1]:51301 req_id = a5f113e9-e9ae-4142-8d77-7cd2e75be74f
141912025-04-30T06:44:22.402ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
141922025-04-30T06:44:22.413ZDEBGtest_utilization (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory result = Ok(())
141932025-04-30T06:44:22.413ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
141942025-04-30T06:44:22.414ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
141952025-04-30T06:44:22.416ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
141962025-04-30T06:44:22.416ZDEBGtest_utilization (BackgroundTasks): begin collection background_task = inventory_collection
141972025-04-30T06:44:22.416ZDEBGtest_utilization (BackgroundTasks): begin collection from MGS background_task = inventory_collection mgs_url = http://[::1]:48759
141982025-04-30T06:44:22.416ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/ignition
141992025-04-30T06:44:22.417ZDEBGtest_utilization (clickhouse-client): inserted rows into table collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 4d1485f0-110e-4161-9a33-c4610d68be5d n_rows = 2 table_name = fields_string
142002025-04-30T06:44:22.417ZTRACtest_utilization (clickhouse-client): inserting data collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 4d1485f0-110e-4161-9a33-c4610d68be5d n_columns = 4 n_rows = 1 sql = INSERT INTO oximeter.measurements_i64 FORMAT Native
142012025-04-30T06:44:22.423ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
142022025-04-30T06:44:22.423ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
142032025-04-30T06:44:22.424ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
142042025-04-30T06:44:22.424ZDEBGtest_utilization (clickhouse-client): inserted rows into table collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 4d1485f0-110e-4161-9a33-c4610d68be5d n_rows = 1 table_name = measurements_i64
142052025-04-30T06:44:22.424ZTRACtest_utilization (results-sink): successfully inserted samples collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
142062025-04-30T06:44:22.431ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/ignition", status: 200, headers: {"content-type": "application/json", "x-request-id": "5efde4e8-5124-4f87-93e9-96c5b62e703f", "content-length": "835", "date": "Wed, 30 Apr 2025 06:44:22 GMT"} })
142072025-04-30T06:44:22.431ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0
142082025-04-30T06:44:22.437ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "fed5b14d-93cd-485a-aef5-fc463eb61f62", "content-length": "734", "date": "Wed, 30 Apr 2025 06:44:22 GMT"} })
142092025-04-30T06:44:22.437ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0/component/sp/caboose?firmware_slot=0
142102025-04-30T06:44:22.457ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "9560c70a-5bb8-4fcd-8fb0-208563913b08", "content-length": "111", "date": "Wed, 30 Apr 2025 06:44:22 GMT"} })
142112025-04-30T06:44:22.458ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0/component/sp/caboose?firmware_slot=1
142122025-04-30T06:44:22.463ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "d19fc7b1-1e3a-41e0-a96a-0354e4bc6f10", "content-length": "111", "date": "Wed, 30 Apr 2025 06:44:22 GMT"} })
142132025-04-30T06:44:22.463ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0/component/rot/caboose?firmware_slot=0
142142025-04-30T06:44:22.477ZDEBGtest_utilization: attempt to fetch latest inventory collection result = Ok("found")
142152025-04-30T06:44:22.483ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "62f5b92c-17d1-4cfb-93ed-9627a1510dda", "content-length": "170", "date": "Wed, 30 Apr 2025 06:44:22 GMT"} })
142162025-04-30T06:44:22.483ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0/component/rot/caboose?firmware_slot=1
142172025-04-30T06:44:22.486ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "0bcc09b2-cd42-4e7f-9117-8e93f58952ff", "content-length": "170", "date": "Wed, 30 Apr 2025 06:44:22 GMT"} })
142182025-04-30T06:44:22.486ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0/component/stage0/caboose?firmware_slot=0
142192025-04-30T06:44:22.495ZWARNtest_utilization (results-sink-cluster): failed to insert some results into metric DB collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 error = Failed to check out connection to database: No backends found for this service file = oximeter/collector/src/results_sink.rs:92
142202025-04-30T06:44:22.496ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "9f03c0b3-d03a-4793-8e3e-9da7eb0efb54", "content-length": "179", "date": "Wed, 30 Apr 2025 06:44:22 GMT"} })
142212025-04-30T06:44:22.496ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0/component/stage0/caboose?firmware_slot=1
142222025-04-30T06:44:22.499ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "959ff7c1-6b0a-4ea6-af47-645f70795ad1", "content-length": "179", "date": "Wed, 30 Apr 2025 06:44:22 GMT"} })
142232025-04-30T06:44:22.499ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0/component/rot/cmpa
142242025-04-30T06:44:22.500ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "b2bee2f2-c4ee-4b0a-944f-b40437c56f81", "content-length": "702", "date": "Wed, 30 Apr 2025 06:44:22 GMT"} })
142252025-04-30T06:44:22.501ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0/component/rot/cfpa
142262025-04-30T06:44:22.503ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "4c665297-a8ce-47c2-950a-803dd54b6c48", "content-length": "727", "date": "Wed, 30 Apr 2025 06:44:22 GMT"} })
142272025-04-30T06:44:22.503ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0/component/rot/cfpa
142282025-04-30T06:44:22.506ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "9540a318-2263-4551-8923-04f566b11e06", "content-length": "729", "date": "Wed, 30 Apr 2025 06:44:22 GMT"} })
142292025-04-30T06:44:22.506ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0/component/rot/cfpa
142302025-04-30T06:44:22.508ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "e3b94601-1bb5-4887-8053-b1e96b324c37", "content-length": "728", "date": "Wed, 30 Apr 2025 06:44:22 GMT"} })
142312025-04-30T06:44:22.509ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1
142322025-04-30T06:44:22.511ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1", status: 200, headers: {"content-type": "application/json", "x-request-id": "0022d0e1-1248-49bf-b482-19bf9b1d2369", "content-length": "734", "date": "Wed, 30 Apr 2025 06:44:22 GMT"} })
142332025-04-30T06:44:22.512ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1/component/sp/caboose?firmware_slot=0
142342025-04-30T06:44:22.513ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "2423b8f9-4196-4bde-9a51-6763d959ea23", "content-length": "111", "date": "Wed, 30 Apr 2025 06:44:22 GMT"} })
142352025-04-30T06:44:22.514ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1/component/sp/caboose?firmware_slot=1
142362025-04-30T06:44:22.516ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "ffc8beab-da0b-4cbc-8874-d9f44acb95ed", "content-length": "111", "date": "Wed, 30 Apr 2025 06:44:22 GMT"} })
142372025-04-30T06:44:22.516ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1/component/rot/caboose?firmware_slot=0
142382025-04-30T06:44:22.519ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "0b79b8bb-8f44-4958-906a-76dcb7d69343", "content-length": "170", "date": "Wed, 30 Apr 2025 06:44:22 GMT"} })
142392025-04-30T06:44:22.519ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1/component/rot/caboose?firmware_slot=1
142402025-04-30T06:44:22.521ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "6ecbc0e0-d069-4327-84d8-c94d24d3080a", "content-length": "170", "date": "Wed, 30 Apr 2025 06:44:22 GMT"} })
142412025-04-30T06:44:22.521ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1/component/stage0/caboose?firmware_slot=0
142422025-04-30T06:44:22.523ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "347d25e0-952c-439d-a904-7ed96e70aaa5", "content-length": "179", "date": "Wed, 30 Apr 2025 06:44:22 GMT"} })
142432025-04-30T06:44:22.523ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1/component/stage0/caboose?firmware_slot=1
142442025-04-30T06:44:22.532ZTRACtest_utilization (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory
142452025-04-30T06:44:22.532ZTRACtest_utilization (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
142462025-04-30T06:44:22.532ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
142472025-04-30T06:44:22.532ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
142482025-04-30T06:44:22.535ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
142492025-04-30T06:44:22.538ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "b0a7770e-39c0-484c-8546-b347bd6470a7", "content-length": "179", "date": "Wed, 30 Apr 2025 06:44:22 GMT"} })
142502025-04-30T06:44:22.539ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1/component/rot/cmpa
142512025-04-30T06:44:22.543ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "cd62012a-eabb-46fb-bd90-69b84d7a1a8d", "content-length": "702", "date": "Wed, 30 Apr 2025 06:44:22 GMT"} })
142522025-04-30T06:44:22.543ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1/component/rot/cfpa
142532025-04-30T06:44:22.545ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
142542025-04-30T06:44:22.556ZDEBGtest_utilization (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory result = Ok(())
142552025-04-30T06:44:22.556ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
142562025-04-30T06:44:22.556ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
142572025-04-30T06:44:22.557ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
142582025-04-30T06:44:22.560ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "153b4222-60e3-450e-9c26-2867d5c94a44", "content-length": "727", "date": "Wed, 30 Apr 2025 06:44:22 GMT"} })
142592025-04-30T06:44:22.560ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1/component/rot/cfpa
142602025-04-30T06:44:22.560ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
142612025-04-30T06:44:22.560ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
142622025-04-30T06:44:22.561ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
142632025-04-30T06:44:22.567ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "fd5d615c-c1a1-46e2-bfdc-5cf96f2cb89c", "content-length": "729", "date": "Wed, 30 Apr 2025 06:44:22 GMT"} })
142642025-04-30T06:44:22.567ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1/component/rot/cfpa
142652025-04-30T06:44:22.579ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "6a5300a1-5750-453c-ae5a-eecb93e73820", "content-length": "728", "date": "Wed, 30 Apr 2025 06:44:22 GMT"} })
142662025-04-30T06:44:22.596ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0
142672025-04-30T06:44:22.648ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "cd855305-da5e-47bc-b84c-e21c18a79477", "content-length": "723", "date": "Wed, 30 Apr 2025 06:44:22 GMT"} })
142682025-04-30T06:44:22.648ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0/component/sp/caboose?firmware_slot=0
142692025-04-30T06:44:22.726ZDEBGtest_utilization: attempt to fetch latest inventory collection result = Ok("found")
142702025-04-30T06:44:22.729ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "f85fdb63-09a1-4692-afdb-392748503df4", "content-length": "109", "date": "Wed, 30 Apr 2025 06:44:22 GMT"} })
142712025-04-30T06:44:22.731ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0/component/sp/caboose?firmware_slot=1
142722025-04-30T06:44:22.738ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "bcb0bf43-3980-48d5-bbc5-bd6e06bf33d0", "content-length": "109", "date": "Wed, 30 Apr 2025 06:44:22 GMT"} })
142732025-04-30T06:44:22.739ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0/component/rot/caboose?firmware_slot=0
142742025-04-30T06:44:22.741ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "2ab4d0a9-b857-4653-8e2f-6cc875d86c6a", "content-length": "169", "date": "Wed, 30 Apr 2025 06:44:22 GMT"} })
142752025-04-30T06:44:22.741ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0/component/rot/caboose?firmware_slot=1
142762025-04-30T06:44:22.745ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "5feba1d0-cdc1-4435-bc83-7102fd652510", "content-length": "169", "date": "Wed, 30 Apr 2025 06:44:22 GMT"} })
142772025-04-30T06:44:22.745ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0/component/stage0/caboose?firmware_slot=0
142782025-04-30T06:44:22.756ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "f973ff0c-b44f-44f5-a366-0a942ba1ac71", "content-length": "178", "date": "Wed, 30 Apr 2025 06:44:22 GMT"} })
142792025-04-30T06:44:22.756ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0/component/stage0/caboose?firmware_slot=1
142802025-04-30T06:44:22.764ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "79c13ea6-c31f-45fd-abdd-ceccd61dbd8b", "content-length": "178", "date": "Wed, 30 Apr 2025 06:44:22 GMT"} })
142812025-04-30T06:44:22.765ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0/component/rot/cmpa
142822025-04-30T06:44:22.770ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "6a55d131-a2c3-4ca4-b6d3-777180abf145", "content-length": "702", "date": "Wed, 30 Apr 2025 06:44:22 GMT"} })
142832025-04-30T06:44:22.770ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0/component/rot/cfpa
142842025-04-30T06:44:22.777ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "f0849366-8203-4c05-a41a-3be4411de0d4", "content-length": "727", "date": "Wed, 30 Apr 2025 06:44:22 GMT"} })
142852025-04-30T06:44:22.777ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0/component/rot/cfpa
142862025-04-30T06:44:22.777ZTRACtest_utilization (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory
142872025-04-30T06:44:22.777ZTRACtest_utilization (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
142882025-04-30T06:44:22.778ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
142892025-04-30T06:44:22.778ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
142902025-04-30T06:44:22.792ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
142912025-04-30T06:44:22.795ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "e2b6c6a3-0c31-45a6-be22-6f7f55d15125", "content-length": "729", "date": "Wed, 30 Apr 2025 06:44:22 GMT"} })
142922025-04-30T06:44:22.795ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0/component/rot/cfpa
142932025-04-30T06:44:22.798ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
142942025-04-30T06:44:22.813ZDEBGtest_utilization (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory result = Ok(())
142952025-04-30T06:44:22.813ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
142962025-04-30T06:44:22.813ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
142972025-04-30T06:44:22.814ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
142982025-04-30T06:44:22.814ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "d5d606f4-0783-4cb8-b7c2-0cc0c0709dcb", "content-length": "728", "date": "Wed, 30 Apr 2025 06:44:22 GMT"} })
142992025-04-30T06:44:22.815ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1
143002025-04-30T06:44:22.820ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
143012025-04-30T06:44:22.820ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
143022025-04-30T06:44:22.822ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
143032025-04-30T06:44:22.823ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1", status: 200, headers: {"content-type": "application/json", "x-request-id": "a1ae86da-968a-423d-a0ab-633a0076f385", "content-length": "723", "date": "Wed, 30 Apr 2025 06:44:22 GMT"} })
143042025-04-30T06:44:22.823ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1/component/sp/caboose?firmware_slot=0
143052025-04-30T06:44:22.829ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "79ba6329-8ba6-4eac-b092-79c46e0b9e45", "content-length": "109", "date": "Wed, 30 Apr 2025 06:44:22 GMT"} })
143062025-04-30T06:44:22.829ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1/component/sp/caboose?firmware_slot=1
143072025-04-30T06:44:22.833ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "87ddf98f-829e-4ccd-aeda-3cd9c6ee260c", "content-length": "109", "date": "Wed, 30 Apr 2025 06:44:22 GMT"} })
143082025-04-30T06:44:22.833ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1/component/rot/caboose?firmware_slot=0
143092025-04-30T06:44:22.837ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "4a826f7b-3ccf-45d7-92a2-69c11aba02d4", "content-length": "169", "date": "Wed, 30 Apr 2025 06:44:22 GMT"} })
143102025-04-30T06:44:22.839ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1/component/rot/caboose?firmware_slot=1
143112025-04-30T06:44:22.843ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "93aa76f3-1b60-4cd4-8ab1-5c6cef89b173", "content-length": "169", "date": "Wed, 30 Apr 2025 06:44:22 GMT"} })
143122025-04-30T06:44:22.844ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1/component/stage0/caboose?firmware_slot=0
143132025-04-30T06:44:22.849ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "a0a6a914-7948-4d99-a47f-da9a58af5b62", "content-length": "178", "date": "Wed, 30 Apr 2025 06:44:22 GMT"} })
143142025-04-30T06:44:22.849ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1/component/stage0/caboose?firmware_slot=1
143152025-04-30T06:44:22.853ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "6fbabf1b-39c1-4699-b0e7-4fbfa4a868e4", "content-length": "178", "date": "Wed, 30 Apr 2025 06:44:22 GMT"} })
143162025-04-30T06:44:22.854ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1/component/rot/cmpa
143172025-04-30T06:44:22.860ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "bf5d01ec-99c1-4fd4-8dd2-780fd9ace78c", "content-length": "702", "date": "Wed, 30 Apr 2025 06:44:22 GMT"} })
143182025-04-30T06:44:22.860ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1/component/rot/cfpa
143192025-04-30T06:44:22.861ZDEBGtest_utilization: attempt to fetch latest inventory collection result = Ok("found")
143202025-04-30T06:44:22.862ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "398bd46d-0c60-4660-9f1c-ce8ee1ed2069", "content-length": "727", "date": "Wed, 30 Apr 2025 06:44:22 GMT"} })
143212025-04-30T06:44:22.862ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1/component/rot/cfpa
143222025-04-30T06:44:22.863ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "ddff511b-bf5a-4001-b721-bce5506d49eb", "content-length": "729", "date": "Wed, 30 Apr 2025 06:44:22 GMT"} })
143232025-04-30T06:44:22.863ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1/component/rot/cfpa
143242025-04-30T06:44:22.864ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "d9dfe7d6-d35e-4bae-9fec-fecb1bc84f02", "content-length": "728", "date": "Wed, 30 Apr 2025 06:44:22 GMT"} })
143252025-04-30T06:44:22.864ZTRACtest_utilization (BackgroundTasks): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet
143262025-04-30T06:44:22.864ZTRACtest_utilization (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
143272025-04-30T06:44:22.864ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
143282025-04-30T06:44:22.864ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
143292025-04-30T06:44:22.865ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
143302025-04-30T06:44:22.869ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
143312025-04-30T06:44:22.873ZDEBGtest_utilization (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet result = Ok(())
143322025-04-30T06:44:22.873ZTRACtest_utilization (BackgroundTasks): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet
143332025-04-30T06:44:22.873ZTRACtest_utilization (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
143342025-04-30T06:44:22.874ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
143352025-04-30T06:44:22.874ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
143362025-04-30T06:44:22.874ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
143372025-04-30T06:44:22.881ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
143382025-04-30T06:44:22.886ZDEBGtest_utilization (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet result = Ok(())
143392025-04-30T06:44:22.886ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
143402025-04-30T06:44:22.886ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
143412025-04-30T06:44:22.886ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
143422025-04-30T06:44:22.966ZDEBGtest_utilization (BackgroundTasks): begin collection from Sled Agent background_task = inventory_collection sled_agent_url = http://[::1]:39365
143432025-04-30T06:44:22.966ZDEBGtest_utilization (BackgroundTasks): client request SledAgent = http://[::1]:39365 background_task = inventory_collection body = None method = GET uri = http://[::1]:39365/inventory
143442025-04-30T06:44:22.967ZTRACtest_utilization (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory
143452025-04-30T06:44:22.967ZTRACtest_utilization (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
143462025-04-30T06:44:22.967ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
143472025-04-30T06:44:22.967ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
143482025-04-30T06:44:22.968ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
143492025-04-30T06:44:22.968ZINFOtest_utilization (dropshot): accepted connection local_addr = [::1]:39365 remote_addr = [::1]:35245 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
143502025-04-30T06:44:22.969ZTRACtest_utilization (dropshot): incoming request local_addr = [::1]:39365 method = GET remote_addr = [::1]:35245 req_id = 240a0ec1-79d9-402d-b3f9-c9e9318aa46c sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /inventory
143512025-04-30T06:44:22.969ZINFOtest_utilization (dropshot): request completed latency_us = 675 local_addr = [::1]:39365 method = GET remote_addr = [::1]:35245 req_id = 240a0ec1-79d9-402d-b3f9-c9e9318aa46c response_code = 200 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /inventory
143522025-04-30T06:44:22.970ZDEBGtest_utilization (BackgroundTasks): client response SledAgent = http://[::1]:39365 background_task = inventory_collection result = Ok(Response { url: "http://[::1]:39365/inventory", status: 200, headers: {"content-type": "application/json", "x-request-id": "240a0ec1-79d9-402d-b3f9-c9e9318aa46c", "content-length": "3141", "date": "Wed, 30 Apr 2025 06:44:22 GMT"} })
143532025-04-30T06:44:22.972ZDEBGtest_utilization (BackgroundTasks): begin collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
143542025-04-30T06:44:22.972ZDEBGtest_utilization (BackgroundTasks): end collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
143552025-04-30T06:44:22.972ZDEBGtest_utilization (BackgroundTasks): finished collection background_task = inventory_collection
143562025-04-30T06:44:22.974ZTRACtest_utilization (BackgroundTasks): authorize begin action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory
143572025-04-30T06:44:22.974ZTRACtest_utilization (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
143582025-04-30T06:44:22.974ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
143592025-04-30T06:44:22.974ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
143602025-04-30T06:44:22.975ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
143612025-04-30T06:44:22.976ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
143622025-04-30T06:44:23.007ZDEBGtest_utilization (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory result = Ok(())
143632025-04-30T06:44:23.008ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
143642025-04-30T06:44:23.008ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
143652025-04-30T06:44:23.008ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
143662025-04-30T06:44:23.011ZDEBGtest_utilization (collection-task): sent timer-based collection request to the collection task collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
143672025-04-30T06:44:23.011ZDEBGtest_utilization (collection-task): collecting from producer collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
143682025-04-30T06:44:23.015ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
143692025-04-30T06:44:23.023ZDEBGtest_utilization (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory result = Ok(())
143702025-04-30T06:44:23.023ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
143712025-04-30T06:44:23.023ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
143722025-04-30T06:44:23.024ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
143732025-04-30T06:44:23.025ZTRACtest_utilization (collection-task): sent collection request to producer collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
143742025-04-30T06:44:23.026ZDEBGtest_utilization (collection-task): collected results from producer collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 n_results = 1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
143752025-04-30T06:44:23.026ZTRACtest_utilization (collection-task): forwarded results to main collection loop collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
143762025-04-30T06:44:23.026ZTRACtest_utilization (collection-task): top of inner collection loop, waiting for next request collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
143772025-04-30T06:44:23.027ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
143782025-04-30T06:44:23.028ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
143792025-04-30T06:44:23.028ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
143802025-04-30T06:44:23.059ZWARNtest_utilization (datastore): Non-retryable transaction failure err = DatabaseError(UniqueViolation, "duplicate key value violates unique constraint \\"inv_caboose_pkey\\"") transaction = inventory_insert_collection
143812025-04-30T06:44:23.060ZWARNtest_utilization (BackgroundTasks): inventory collection failed background_task = inventory_collection error = failed to collect inventory: saving inventory to database: Internal Error: unexpected database error: duplicate key value violates unique constraint "inv_caboose_pkey"
143822025-04-30T06:44:23.060ZDEBGtest_utilization (BackgroundTasks): activation complete background_task = inventory_collection elapsed = 821.789371ms iteration = 39 status = {"error":"failed to collect inventory: saving inventory to database: Internal Error: unexpected database error: duplicate key value violates unique constraint \\"inv_caboose_pkey\\""}
143832025-04-30T06:44:23.060ZDEBGtest_utilization (BackgroundTasks): activating background_task = inventory_collection iteration = 40 reason = Signaled
143842025-04-30T06:44:23.060ZTRACtest_utilization (BackgroundTasks): authorize begin action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory
143852025-04-30T06:44:23.060ZTRACtest_utilization (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
143862025-04-30T06:44:23.060ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
143872025-04-30T06:44:23.060ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
143882025-04-30T06:44:23.061ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
143892025-04-30T06:44:23.067ZDEBGtest_utilization: attempt to fetch latest inventory collection result = Ok("found")
143902025-04-30T06:44:23.068ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
143912025-04-30T06:44:23.082ZDEBGtest_utilization (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory result = Ok(())
143922025-04-30T06:44:23.082ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
143932025-04-30T06:44:23.082ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
143942025-04-30T06:44:23.082ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
143952025-04-30T06:44:23.086ZDEBGtest_utilization (BackgroundTasks): inventory_prune_one: nothing eligible for removal (too few) background_task = inventory_collection candidates = [(13d8b8d9-6c6b-47a6-9190-952544edb75b, 0)]
143962025-04-30T06:44:23.087ZTRACtest_utilization (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal
143972025-04-30T06:44:23.087ZTRACtest_utilization (dns): buffer buffer = Length: 67 (0x43) bytes\n0000: e6 3c 01 00 00 01 00 00 00 00 00 01 04 5f 6d 67 .<..........._mg\n0010: 73 04 5f 74 63 70 0d 63 6f 6e 74 72 6f 6c 2d 70 s._tcp.control-p\n0020: 6c 61 6e 65 05 6f 78 69 64 65 08 69 6e 74 65 72 lane.oxide.inter\n0030: 6e 61 6c 00 00 21 00 01 00 00 29 04 d0 00 00 00 nal..!....).....\n0040: 00 00 00 ... kind = dns peer_addr = [::1]:58539 req_id = e719e07e-7e0e-4fdc-93b8-8787014ee8f7
143982025-04-30T06:44:23.087ZDEBGtest_utilization (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 58940,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_mgs._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_mgs._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 4,\n 95,\n 109,\n 103,\n 115,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:58539 req_id = e719e07e-7e0e-4fdc-93b8-8787014ee8f7
143992025-04-30T06:44:23.087ZDEBGtest_utilization (store): query key key = _mgs._tcp
144002025-04-30T06:44:23.087ZDEBGtest_utilization (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
144012025-04-30T06:44:23.088ZDEBGtest_utilization (dns): dns response additional_records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal"), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] kind = dns peer_addr = [::1]:58539 query = LowerQuery { name: LowerName(Name("_mgs._tcp.control-plane.oxide.internal.")), original: Query { name: Name("_mgs._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN } } records = [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 48759, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal") })) }] req_id = e719e07e-7e0e-4fdc-93b8-8787014ee8f7
144022025-04-30T06:44:23.088ZDEBGtest_utilization (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgs._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 48759, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }], valid_until: Instant { tv_sec: 4327, tv_nsec: 887377070 } })
144032025-04-30T06:44:23.088ZTRACtest_utilization (DnsResolver): lookup_service_targets: looking up SRV target
144042025-04-30T06:44:23.089ZTRACtest_utilization (dns): buffer buffer = Length: 108 (0x6c) bytes\n0000: f0 92 01 00 00 01 00 00 00 00 00 01 2d 64 65 6e ............-den\n0010: 64 72 69 74 65 2d 62 36 64 36 35 33 34 31 2d 31 drite-b6d65341-1\n0020: 36 37 63 2d 34 31 64 66 2d 39 62 35 63 2d 34 31 67c-41df-9b5c-41\n0030: 63 64 65 64 39 39 63 32 32 39 04 68 6f 73 74 0d cded99c229.host.\n0040: 63 6f 6e 74 72 6f 6c 2d 70 6c 61 6e 65 05 6f 78 control-plane.ox\n0050: 69 64 65 08 69 6e 74 65 72 6e 61 6c 00 00 1c 00 ide.internal....\n0060: 01 00 00 29 04 d0 00 00 00 00 00 00 ...)........ kind = dns peer_addr = [::1]:57272 req_id = cb458d39-e11a-43e1-9ec3-6c7bc390b2bd
144052025-04-30T06:44:23.089ZDEBGtest_utilization (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 61586,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n query_type: AAAA,\n query_class: IN,\n },\n },\n original: [\n 45,\n 100,\n 101,\n 110,\n 100,\n 114,\n 105,\n 116,\n 101,\n 45,\n 98,\n 54,\n 100,\n 54,\n 53,\n 51,\n 52,\n 49,\n 45,\n 49,\n 54,\n 55,\n 99,\n 45,\n 52,\n 49,\n 100,\n 102,\n 45,\n 57,\n 98,\n 53,\n 99,\n 45,\n 52,\n 49,\n 99,\n 100,\n 101,\n 100,\n 57,\n 57,\n 99,\n 50,\n 50,\n 57,\n 4,\n 104,\n 111,\n 115,\n 116,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 28,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:57272 req_id = cb458d39-e11a-43e1-9ec3-6c7bc390b2bd
144062025-04-30T06:44:23.089ZDEBGtest_utilization (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
144072025-04-30T06:44:23.089ZDEBGtest_utilization (dns): dns response additional_records = [] kind = dns peer_addr = [::1]:57272 query = LowerQuery { name: LowerName(Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.")), original: Query { name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), query_type: AAAA, query_class: IN } } records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] req_id = cb458d39-e11a-43e1-9ec3-6c7bc390b2bd
144082025-04-30T06:44:23.153ZTRACtest_utilization (DnsResolver): lookup_all_socket_v6 srv dns_name = _clickhouse-admin-keeper._tcp.control-plane.oxide.internal
144092025-04-30T06:44:23.153ZTRACtest_utilization (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory
144102025-04-30T06:44:23.154ZTRACtest_utilization (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
144112025-04-30T06:44:23.154ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
144122025-04-30T06:44:23.154ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
144132025-04-30T06:44:23.155ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
144142025-04-30T06:44:23.155ZTRACtest_utilization (dns): buffer buffer = Length: 87 (0x57) bytes\n0000: dd 29 01 00 00 01 00 00 00 00 00 01 18 5f 63 6c .)..........._cl\n0010: 69 63 6b 68 6f 75 73 65 2d 61 64 6d 69 6e 2d 6b ickhouse-admin-k\n0020: 65 65 70 65 72 04 5f 74 63 70 0d 63 6f 6e 74 72 eeper._tcp.contr\n0030: 6f 6c 2d 70 6c 61 6e 65 05 6f 78 69 64 65 08 69 ol-plane.oxide.i\n0040: 6e 74 65 72 6e 61 6c 00 00 21 00 01 00 00 29 04 nternal..!....).\n0050: d0 00 00 00 00 00 00 ....... kind = dns peer_addr = [::1]:59743 req_id = 32be437c-b7ba-432d-be1c-d28f76cee856
144152025-04-30T06:44:23.155ZDEBGtest_utilization (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 56617,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 24,\n 95,\n 99,\n 108,\n 105,\n 99,\n 107,\n 104,\n 111,\n 117,\n 115,\n 101,\n 45,\n 97,\n 100,\n 109,\n 105,\n 110,\n 45,\n 107,\n 101,\n 101,\n 112,\n 101,\n 114,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:59743 req_id = 32be437c-b7ba-432d-be1c-d28f76cee856
144162025-04-30T06:44:23.155ZDEBGtest_utilization (store): query key key = _clickhouse-admin-keeper._tcp
144172025-04-30T06:44:23.155ZERROtest_utilization (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 56617, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 1, }, query: WireQuery { query: LowerQuery { name: LowerName( Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), ), original: Query { name: Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN, }, }, original: [ 24, 95, 99, 108, 105, 99, 107, 104, 111, 117, 115, 101, 45, 97, 100, 109, 105, 110, 45, 107, 101, 101, 112, 101, 114, 4, 95, 116, 99, 112, 13, 99, 111, 110, 116, 114, 111, 108, 45, 112, 108, 97, 110, 101, 5, 111, 120, 105, 100, 101, 8, 105, 110, 116, 101, 114, 110, 97, 108, 0, 0, 33, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: Some( Edns { rcode_high: 0, version: 0, dnssec_ok: false, max_payload: 1232, options: OPT { options: {}, }, }, ), } NXDOMAIN: no records found for name: "_clickhouse-admin-keeper._tcp.control-plane.oxide.internal." kind = dns peer_addr = [::1]:59743 req_id = 32be437c-b7ba-432d-be1c-d28f76cee856
144182025-04-30T06:44:23.156ZDEBGtest_utilization (BackgroundTasks): begin collection background_task = inventory_collection
144192025-04-30T06:44:23.156ZDEBGtest_utilization (BackgroundTasks): begin collection from MGS background_task = inventory_collection mgs_url = http://[::1]:48759
144202025-04-30T06:44:23.156ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/ignition
144212025-04-30T06:44:23.158ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
144222025-04-30T06:44:23.165ZDEBGtest_utilization (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory result = Ok(())
144232025-04-30T06:44:23.165ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
144242025-04-30T06:44:23.165ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
144252025-04-30T06:44:23.166ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
144262025-04-30T06:44:23.167ZTRACtest_utilization (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal
144272025-04-30T06:44:23.169ZTRACtest_utilization (dns): buffer buffer = Length: 72 (0x48) bytes\n0000: c0 3f 01 00 00 01 00 00 00 00 00 01 09 5f 64 65 .?..........._de\n0010: 6e 64 72 69 74 65 04 5f 74 63 70 0d 63 6f 6e 74 ndrite._tcp.cont\n0020: 72 6f 6c 2d 70 6c 61 6e 65 05 6f 78 69 64 65 08 rol-plane.oxide.\n0030: 69 6e 74 65 72 6e 61 6c 00 00 21 00 01 00 00 29 internal..!....)\n0040: 04 d0 00 00 00 00 00 00 ........ kind = dns peer_addr = [::1]:54331 req_id = 126d8de6-3a6d-4473-9102-d6fe1812bd37
144282025-04-30T06:44:23.169ZDEBGtest_utilization (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 49215,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_dendrite._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_dendrite._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 9,\n 95,\n 100,\n 101,\n 110,\n 100,\n 114,\n 105,\n 116,\n 101,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:54331 req_id = 126d8de6-3a6d-4473-9102-d6fe1812bd37
144292025-04-30T06:44:23.170ZDEBGtest_utilization (store): query key key = _dendrite._tcp
144302025-04-30T06:44:23.170ZDEBGtest_utilization (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
144312025-04-30T06:44:23.170ZDEBGtest_utilization (dns): dns response additional_records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal"), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] kind = dns peer_addr = [::1]:54331 query = LowerQuery { name: LowerName(Name("_dendrite._tcp.control-plane.oxide.internal.")), original: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN } } records = [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 47974, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal") })) }] req_id = 126d8de6-3a6d-4473-9102-d6fe1812bd37
144322025-04-30T06:44:23.171ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
144332025-04-30T06:44:23.171ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
144342025-04-30T06:44:23.172ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
144352025-04-30T06:44:23.172ZDEBGtest_utilization (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 47974, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }], valid_until: Instant { tv_sec: 4327, tv_nsec: 971793314 } })
144362025-04-30T06:44:23.172ZTRACtest_utilization (DnsResolver): lookup_service_targets: looking up SRV target
144372025-04-30T06:44:23.173ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/ignition", status: 200, headers: {"content-type": "application/json", "x-request-id": "ebc5c232-73ed-4340-8a29-525230ca8af9", "content-length": "835", "date": "Wed, 30 Apr 2025 06:44:23 GMT"} })
144382025-04-30T06:44:23.173ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0
144392025-04-30T06:44:23.175ZTRACtest_utilization (dns): buffer buffer = Length: 108 (0x6c) bytes\n0000: db c2 01 00 00 01 00 00 00 00 00 01 2d 64 65 6e ............-den\n0010: 64 72 69 74 65 2d 62 36 64 36 35 33 34 31 2d 31 drite-b6d65341-1\n0020: 36 37 63 2d 34 31 64 66 2d 39 62 35 63 2d 34 31 67c-41df-9b5c-41\n0030: 63 64 65 64 39 39 63 32 32 39 04 68 6f 73 74 0d cded99c229.host.\n0040: 63 6f 6e 74 72 6f 6c 2d 70 6c 61 6e 65 05 6f 78 control-plane.ox\n0050: 69 64 65 08 69 6e 74 65 72 6e 61 6c 00 00 1c 00 ide.internal....\n0060: 01 00 00 29 04 d0 00 00 00 00 00 00 ...)........ kind = dns peer_addr = [::1]:55009 req_id = bc7b6b31-94b0-49ff-a7b5-1df2f5686833
144402025-04-30T06:44:23.175ZDEBGtest_utilization (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 56258,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n query_type: AAAA,\n query_class: IN,\n },\n },\n original: [\n 45,\n 100,\n 101,\n 110,\n 100,\n 114,\n 105,\n 116,\n 101,\n 45,\n 98,\n 54,\n 100,\n 54,\n 53,\n 51,\n 52,\n 49,\n 45,\n 49,\n 54,\n 55,\n 99,\n 45,\n 52,\n 49,\n 100,\n 102,\n 45,\n 57,\n 98,\n 53,\n 99,\n 45,\n 52,\n 49,\n 99,\n 100,\n 101,\n 100,\n 57,\n 57,\n 99,\n 50,\n 50,\n 57,\n 4,\n 104,\n 111,\n 115,\n 116,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 28,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:55009 req_id = bc7b6b31-94b0-49ff-a7b5-1df2f5686833
144412025-04-30T06:44:23.176ZDEBGtest_utilization (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
144422025-04-30T06:44:23.176ZDEBGtest_utilization (dns): dns response additional_records = [] kind = dns peer_addr = [::1]:55009 query = LowerQuery { name: LowerName(Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.")), original: Query { name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), query_type: AAAA, query_class: IN } } records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] req_id = bc7b6b31-94b0-49ff-a7b5-1df2f5686833
144432025-04-30T06:44:23.176ZINFOtest_utilization (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_v4_garbage_collector
144442025-04-30T06:44:23.253ZINFOtest_utilization (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_v4_garbage_collector zone_address = ::1
144452025-04-30T06:44:23.253ZDEBGtest_utilization (MgsClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[::1]:12225/local/switch-id
144462025-04-30T06:44:23.253ZDEBGtest_utilization (MgsClient): client response background_task = nat_v4_garbage_collector result = Err(reqwest::Error { kind: Request, url: "http://[::1]:12225/local/switch-id", source: hyper_util::client::legacy::Error(Connect, ConnectError("tcp connect error", Os { code: 146, kind: ConnectionRefused, message: "Connection refused" })) })
144472025-04-30T06:44:23.253ZWARNtest_utilization (BackgroundTasks): failed to identify switch slot for dendrite background_task = nat_v4_garbage_collector reason = Communication Error: error sending request for url (http://[::1]:12225/local/switch-id): client error (Connect): tcp connect error: Connection refused (os error 146): Connection refused (os error 146) zone_address = ::1
144482025-04-30T06:44:23.260ZWARNtest_utilization (BackgroundTasks): Failed to map switch zone addr: Communication Error: error sending request for url (http://[::1]:12225/local/switch-id), retrying background_task = nat_v4_garbage_collector
144492025-04-30T06:44:23.261ZTRACtest_utilization (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal
144502025-04-30T06:44:23.262ZTRACtest_utilization (dns): buffer buffer = Length: 72 (0x48) bytes\n0000: 9d 2b 01 00 00 01 00 00 00 00 00 01 09 5f 64 65 .+..........._de\n0010: 6e 64 72 69 74 65 04 5f 74 63 70 0d 63 6f 6e 74 ndrite._tcp.cont\n0020: 72 6f 6c 2d 70 6c 61 6e 65 05 6f 78 69 64 65 08 rol-plane.oxide.\n0030: 69 6e 74 65 72 6e 61 6c 00 00 21 00 01 00 00 29 internal..!....)\n0040: 04 d0 00 00 00 00 00 00 ........ kind = dns peer_addr = [::1]:52138 req_id = 7b4d84da-90cf-478a-96a7-6b55f7ab53a3
144512025-04-30T06:44:23.263ZDEBGtest_utilization (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 40235,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_dendrite._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_dendrite._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 9,\n 95,\n 100,\n 101,\n 110,\n 100,\n 114,\n 105,\n 116,\n 101,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:52138 req_id = 7b4d84da-90cf-478a-96a7-6b55f7ab53a3
144522025-04-30T06:44:23.263ZDEBGtest_utilization (store): query key key = _dendrite._tcp
144532025-04-30T06:44:23.263ZDEBGtest_utilization (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
144542025-04-30T06:44:23.263ZDEBGtest_utilization (dns): dns response additional_records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal"), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] kind = dns peer_addr = [::1]:52138 query = LowerQuery { name: LowerName(Name("_dendrite._tcp.control-plane.oxide.internal.")), original: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN } } records = [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 47974, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal") })) }] req_id = 7b4d84da-90cf-478a-96a7-6b55f7ab53a3
144552025-04-30T06:44:23.263ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "222e606c-3f52-47d7-a32c-05b80ce4a015", "content-length": "734", "date": "Wed, 30 Apr 2025 06:44:23 GMT"} })
144562025-04-30T06:44:23.265ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0/component/sp/caboose?firmware_slot=0
144572025-04-30T06:44:23.267ZDEBGtest_utilization (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 47974, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }], valid_until: Instant { tv_sec: 4328, tv_nsec: 66243830 } })
144582025-04-30T06:44:23.267ZTRACtest_utilization (DnsResolver): lookup_service_targets: looking up SRV target
144592025-04-30T06:44:23.271ZTRACtest_utilization (dns): buffer buffer = Length: 108 (0x6c) bytes\n0000: 31 e1 01 00 00 01 00 00 00 00 00 01 2d 64 65 6e 1...........-den\n0010: 64 72 69 74 65 2d 62 36 64 36 35 33 34 31 2d 31 drite-b6d65341-1\n0020: 36 37 63 2d 34 31 64 66 2d 39 62 35 63 2d 34 31 67c-41df-9b5c-41\n0030: 63 64 65 64 39 39 63 32 32 39 04 68 6f 73 74 0d cded99c229.host.\n0040: 63 6f 6e 74 72 6f 6c 2d 70 6c 61 6e 65 05 6f 78 control-plane.ox\n0050: 69 64 65 08 69 6e 74 65 72 6e 61 6c 00 00 1c 00 ide.internal....\n0060: 01 00 00 29 04 d0 00 00 00 00 00 00 ...)........ kind = dns peer_addr = [::1]:54700 req_id = 0515a2b2-1786-4d60-a991-41f3a7d95c56
144602025-04-30T06:44:23.272ZDEBGtest_utilization (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 12769,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n query_type: AAAA,\n query_class: IN,\n },\n },\n original: [\n 45,\n 100,\n 101,\n 110,\n 100,\n 114,\n 105,\n 116,\n 101,\n 45,\n 98,\n 54,\n 100,\n 54,\n 53,\n 51,\n 52,\n 49,\n 45,\n 49,\n 54,\n 55,\n 99,\n 45,\n 52,\n 49,\n 100,\n 102,\n 45,\n 57,\n 98,\n 53,\n 99,\n 45,\n 52,\n 49,\n 99,\n 100,\n 101,\n 100,\n 57,\n 57,\n 99,\n 50,\n 50,\n 57,\n 4,\n 104,\n 111,\n 115,\n 116,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 28,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:54700 req_id = 0515a2b2-1786-4d60-a991-41f3a7d95c56
144612025-04-30T06:44:23.272ZDEBGtest_utilization (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
144622025-04-30T06:44:23.273ZDEBGtest_utilization (dns): dns response additional_records = [] kind = dns peer_addr = [::1]:54700 query = LowerQuery { name: LowerName(Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.")), original: Query { name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), query_type: AAAA, query_class: IN } } records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] req_id = 0515a2b2-1786-4d60-a991-41f3a7d95c56
144632025-04-30T06:44:23.274ZINFOtest_utilization (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager
144642025-04-30T06:44:23.336ZINFOtest_utilization (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager zone_address = ::1
144652025-04-30T06:44:23.337ZDEBGtest_utilization (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[::1]:12225/local/switch-id
144662025-04-30T06:44:23.337ZDEBGtest_utilization (MgsClient): client response background_task = bfd_manager result = Err(reqwest::Error { kind: Request, url: "http://[::1]:12225/local/switch-id", source: hyper_util::client::legacy::Error(Connect, ConnectError("tcp connect error", Os { code: 146, kind: ConnectionRefused, message: "Connection refused" })) })
144672025-04-30T06:44:23.337ZWARNtest_utilization (BackgroundTasks): failed to identify switch slot for dendrite background_task = bfd_manager reason = Communication Error: error sending request for url (http://[::1]:12225/local/switch-id): client error (Connect): tcp connect error: Connection refused (os error 146): Connection refused (os error 146) zone_address = ::1
144682025-04-30T06:44:23.338ZWARNtest_utilization (BackgroundTasks): Failed to map switch zone addr: Communication Error: error sending request for url (http://[::1]:12225/local/switch-id), retrying background_task = bfd_manager
144692025-04-30T06:44:23.338ZTRACtest_utilization (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal
144702025-04-30T06:44:23.340ZTRACtest_utilization (dns): buffer buffer = Length: 72 (0x48) bytes\n0000: 2d 66 01 00 00 01 00 00 00 00 00 01 09 5f 64 65 -f..........._de\n0010: 6e 64 72 69 74 65 04 5f 74 63 70 0d 63 6f 6e 74 ndrite._tcp.cont\n0020: 72 6f 6c 2d 70 6c 61 6e 65 05 6f 78 69 64 65 08 rol-plane.oxide.\n0030: 69 6e 74 65 72 6e 61 6c 00 00 21 00 01 00 00 29 internal..!....)\n0040: 04 d0 00 00 00 00 00 00 ........ kind = dns peer_addr = [::1]:50346 req_id = f435f83b-f75d-4778-959f-1a03865a3b33
144712025-04-30T06:44:23.340ZDEBGtest_utilization (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 11622,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_dendrite._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_dendrite._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 9,\n 95,\n 100,\n 101,\n 110,\n 100,\n 114,\n 105,\n 116,\n 101,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:50346 req_id = f435f83b-f75d-4778-959f-1a03865a3b33
144722025-04-30T06:44:23.340ZDEBGtest_utilization (store): query key key = _dendrite._tcp
144732025-04-30T06:44:23.340ZDEBGtest_utilization (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
144742025-04-30T06:44:23.340ZDEBGtest_utilization (dns): dns response additional_records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal"), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] kind = dns peer_addr = [::1]:50346 query = LowerQuery { name: LowerName(Name("_dendrite._tcp.control-plane.oxide.internal.")), original: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN } } records = [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 47974, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal") })) }] req_id = f435f83b-f75d-4778-959f-1a03865a3b33
144752025-04-30T06:44:23.341ZDEBGtest_utilization (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 47974, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }], valid_until: Instant { tv_sec: 4328, tv_nsec: 140311425 } })
144762025-04-30T06:44:23.341ZTRACtest_utilization (DnsResolver): lookup_service_targets: looking up SRV target
144772025-04-30T06:44:23.343ZTRACtest_utilization (dns): buffer buffer = Length: 108 (0x6c) bytes\n0000: a1 b4 01 00 00 01 00 00 00 00 00 01 2d 64 65 6e ............-den\n0010: 64 72 69 74 65 2d 62 36 64 36 35 33 34 31 2d 31 drite-b6d65341-1\n0020: 36 37 63 2d 34 31 64 66 2d 39 62 35 63 2d 34 31 67c-41df-9b5c-41\n0030: 63 64 65 64 39 39 63 32 32 39 04 68 6f 73 74 0d cded99c229.host.\n0040: 63 6f 6e 74 72 6f 6c 2d 70 6c 61 6e 65 05 6f 78 control-plane.ox\n0050: 69 64 65 08 69 6e 74 65 72 6e 61 6c 00 00 1c 00 ide.internal....\n0060: 01 00 00 29 04 d0 00 00 00 00 00 00 ...)........ kind = dns peer_addr = [::1]:55810 req_id = f77bc524-79d3-4844-bb16-e27e0d0baea7
144782025-04-30T06:44:23.343ZDEBGtest_utilization (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 41396,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n query_type: AAAA,\n query_class: IN,\n },\n },\n original: [\n 45,\n 100,\n 101,\n 110,\n 100,\n 114,\n 105,\n 116,\n 101,\n 45,\n 98,\n 54,\n 100,\n 54,\n 53,\n 51,\n 52,\n 49,\n 45,\n 49,\n 54,\n 55,\n 99,\n 45,\n 52,\n 49,\n 100,\n 102,\n 45,\n 57,\n 98,\n 53,\n 99,\n 45,\n 52,\n 49,\n 99,\n 100,\n 101,\n 100,\n 57,\n 57,\n 99,\n 50,\n 50,\n 57,\n 4,\n 104,\n 111,\n 115,\n 116,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 28,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:55810 req_id = f77bc524-79d3-4844-bb16-e27e0d0baea7
144792025-04-30T06:44:23.343ZDEBGtest_utilization (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
144802025-04-30T06:44:23.344ZDEBGtest_utilization (dns): dns response additional_records = [] kind = dns peer_addr = [::1]:55810 query = LowerQuery { name: LowerName(Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.")), original: Query { name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), query_type: AAAA, query_class: IN } } records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] req_id = f77bc524-79d3-4844-bb16-e27e0d0baea7
144812025-04-30T06:44:23.344ZINFOtest_utilization (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager rack_id = c19a698f-c6f9-4a17-ae30-20d711b8f7dc
144822025-04-30T06:44:23.428ZINFOtest_utilization (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager rack_id = c19a698f-c6f9-4a17-ae30-20d711b8f7dc zone_address = ::1
144832025-04-30T06:44:23.428ZDEBGtest_utilization (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = c19a698f-c6f9-4a17-ae30-20d711b8f7dc uri = http://[::1]:12225/local/switch-id
144842025-04-30T06:44:23.428ZDEBGtest_utilization (MgsClient): client response background_task = switch_port_config_manager rack_id = c19a698f-c6f9-4a17-ae30-20d711b8f7dc result = Err(reqwest::Error { kind: Request, url: "http://[::1]:12225/local/switch-id", source: hyper_util::client::legacy::Error(Connect, ConnectError("tcp connect error", Os { code: 146, kind: ConnectionRefused, message: "Connection refused" })) })
144852025-04-30T06:44:23.428ZWARNtest_utilization (BackgroundTasks): failed to identify switch slot for dendrite background_task = switch_port_config_manager rack_id = c19a698f-c6f9-4a17-ae30-20d711b8f7dc reason = Communication Error: error sending request for url (http://[::1]:12225/local/switch-id): client error (Connect): tcp connect error: Connection refused (os error 146): Connection refused (os error 146) zone_address = ::1
144862025-04-30T06:44:23.432ZWARNtest_utilization (BackgroundTasks): Failed to map switch zone addr: Communication Error: error sending request for url (http://[::1]:12225/local/switch-id), retrying background_task = switch_port_config_manager rack_id = c19a698f-c6f9-4a17-ae30-20d711b8f7dc
144872025-04-30T06:44:23.432ZDEBGtest_utilization (results-sink): inserting 1 samples into database collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
144882025-04-30T06:44:23.434ZDEBGtest_utilization (results-sink-cluster): inserting 1 samples into database collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
144892025-04-30T06:44:23.434ZDEBGtest_utilization (clickhouse-client): unrolling 1 total samples collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 4d1485f0-110e-4161-9a33-c4610d68be5d
144902025-04-30T06:44:23.435ZTRACtest_utilization (clickhouse-client): inserting data collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 4d1485f0-110e-4161-9a33-c4610d68be5d n_columns = 4 n_rows = 2 sql = INSERT INTO oximeter.fields_string FORMAT Native
144912025-04-30T06:44:23.443ZDEBGtest_utilization (clickhouse-client): inserted rows into table collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 4d1485f0-110e-4161-9a33-c4610d68be5d n_rows = 2 table_name = fields_string
144922025-04-30T06:44:23.443ZTRACtest_utilization (clickhouse-client): inserting data collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 4d1485f0-110e-4161-9a33-c4610d68be5d n_columns = 4 n_rows = 1 sql = INSERT INTO oximeter.measurements_i64 FORMAT Native
144932025-04-30T06:44:23.445ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "751cf4d1-6690-429e-a14e-55f21879a348", "content-length": "111", "date": "Wed, 30 Apr 2025 06:44:23 GMT"} })
144942025-04-30T06:44:23.445ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0/component/sp/caboose?firmware_slot=1
144952025-04-30T06:44:23.451ZDEBGtest_utilization (clickhouse-client): inserted rows into table collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 4d1485f0-110e-4161-9a33-c4610d68be5d n_rows = 1 table_name = measurements_i64
144962025-04-30T06:44:23.451ZTRACtest_utilization (results-sink): successfully inserted samples collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
144972025-04-30T06:44:23.455ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "479eecf2-a2bc-49e0-8c6d-ecde8d2c949f", "content-length": "111", "date": "Wed, 30 Apr 2025 06:44:23 GMT"} })
144982025-04-30T06:44:23.455ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0/component/rot/caboose?firmware_slot=0
144992025-04-30T06:44:23.459ZDEBGtest_utilization: attempt to fetch latest inventory collection result = Ok("found")
145002025-04-30T06:44:23.468ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "11fe7c25-2dbe-4d50-a494-e75265d13df6", "content-length": "170", "date": "Wed, 30 Apr 2025 06:44:23 GMT"} })
145012025-04-30T06:44:23.468ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0/component/rot/caboose?firmware_slot=1
145022025-04-30T06:44:23.473ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "c40c2e55-78c5-403d-9247-076b1b2d1ff5", "content-length": "170", "date": "Wed, 30 Apr 2025 06:44:23 GMT"} })
145032025-04-30T06:44:23.473ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0/component/stage0/caboose?firmware_slot=0
145042025-04-30T06:44:23.477ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "5641dc9c-62fb-49e6-a622-793b69062a06", "content-length": "179", "date": "Wed, 30 Apr 2025 06:44:23 GMT"} })
145052025-04-30T06:44:23.477ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0/component/stage0/caboose?firmware_slot=1
145062025-04-30T06:44:23.481ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "1913fe13-c867-4107-8dd3-fad9d7067931", "content-length": "179", "date": "Wed, 30 Apr 2025 06:44:23 GMT"} })
145072025-04-30T06:44:23.483ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0/component/rot/cmpa
145082025-04-30T06:44:23.486ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "024b8b06-8a0c-433a-80a8-6eaccc9f5ffe", "content-length": "702", "date": "Wed, 30 Apr 2025 06:44:23 GMT"} })
145092025-04-30T06:44:23.486ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0/component/rot/cfpa
145102025-04-30T06:44:23.487ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "274a6afa-ab09-4794-9349-fb0d2d0b80a9", "content-length": "727", "date": "Wed, 30 Apr 2025 06:44:23 GMT"} })
145112025-04-30T06:44:23.487ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0/component/rot/cfpa
145122025-04-30T06:44:23.488ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "94b703fe-3920-4993-9eb9-e5d2af80d953", "content-length": "729", "date": "Wed, 30 Apr 2025 06:44:23 GMT"} })
145132025-04-30T06:44:23.488ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0/component/rot/cfpa
145142025-04-30T06:44:23.490ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "a101479b-05e3-46a6-a0e3-61bc4c155520", "content-length": "728", "date": "Wed, 30 Apr 2025 06:44:23 GMT"} })
145152025-04-30T06:44:23.490ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1
145162025-04-30T06:44:23.493ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1", status: 200, headers: {"content-type": "application/json", "x-request-id": "0b9f2346-617b-4e80-97f0-b6a1f0a3cb71", "content-length": "734", "date": "Wed, 30 Apr 2025 06:44:23 GMT"} })
145172025-04-30T06:44:23.493ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1/component/sp/caboose?firmware_slot=0
145182025-04-30T06:44:23.495ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "a920da54-5011-46d1-874d-9b167fbd799e", "content-length": "111", "date": "Wed, 30 Apr 2025 06:44:23 GMT"} })
145192025-04-30T06:44:23.495ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1/component/sp/caboose?firmware_slot=1
145202025-04-30T06:44:23.499ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "d87c0bfa-31b7-40c7-94a8-492e606d3f7d", "content-length": "111", "date": "Wed, 30 Apr 2025 06:44:23 GMT"} })
145212025-04-30T06:44:23.499ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1/component/rot/caboose?firmware_slot=0
145222025-04-30T06:44:23.505ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "571c4d72-02ff-47e8-ae28-517bf15aec05", "content-length": "170", "date": "Wed, 30 Apr 2025 06:44:23 GMT"} })
145232025-04-30T06:44:23.506ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1/component/rot/caboose?firmware_slot=1
145242025-04-30T06:44:23.509ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "eee24be8-abc0-4a17-87be-61e7d9128093", "content-length": "170", "date": "Wed, 30 Apr 2025 06:44:23 GMT"} })
145252025-04-30T06:44:23.509ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1/component/stage0/caboose?firmware_slot=0
145262025-04-30T06:44:23.512ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "6635290c-86f6-40c3-84f3-b6c68de52198", "content-length": "179", "date": "Wed, 30 Apr 2025 06:44:23 GMT"} })
145272025-04-30T06:44:23.514ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1/component/stage0/caboose?firmware_slot=1
145282025-04-30T06:44:23.520ZTRACtest_utilization (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory
145292025-04-30T06:44:23.520ZTRACtest_utilization (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
145302025-04-30T06:44:23.520ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
145312025-04-30T06:44:23.520ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
145322025-04-30T06:44:23.521ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
145332025-04-30T06:44:23.526ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
145342025-04-30T06:44:23.538ZDEBGtest_utilization (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory result = Ok(())
145352025-04-30T06:44:23.538ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
145362025-04-30T06:44:23.538ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
145372025-04-30T06:44:23.539ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
145382025-04-30T06:44:23.540ZWARNtest_utilization (results-sink-cluster): failed to insert some results into metric DB collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 error = Failed to check out connection to database: No backends found for this service file = oximeter/collector/src/results_sink.rs:92
145392025-04-30T06:44:23.541ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "699987fe-c4b8-47de-984a-3fb9573ff403", "content-length": "179", "date": "Wed, 30 Apr 2025 06:44:23 GMT"} })
145402025-04-30T06:44:23.541ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1/component/rot/cmpa
145412025-04-30T06:44:23.542ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
145422025-04-30T06:44:23.542ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
145432025-04-30T06:44:23.543ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
145442025-04-30T06:44:23.544ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "80a50ea4-b155-4a28-b8f9-3b6ed79a8288", "content-length": "702", "date": "Wed, 30 Apr 2025 06:44:23 GMT"} })
145452025-04-30T06:44:23.544ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1/component/rot/cfpa
145462025-04-30T06:44:23.547ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "691404d0-2f9a-4a04-942a-ed21e84af6d5", "content-length": "727", "date": "Wed, 30 Apr 2025 06:44:23 GMT"} })
145472025-04-30T06:44:23.548ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1/component/rot/cfpa
145482025-04-30T06:44:23.552ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "f56a471f-e86a-4ce8-819b-5cc8c13b88b6", "content-length": "729", "date": "Wed, 30 Apr 2025 06:44:23 GMT"} })
145492025-04-30T06:44:23.552ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1/component/rot/cfpa
145502025-04-30T06:44:23.562ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "f246901d-b519-4952-8876-5ded7f63424a", "content-length": "728", "date": "Wed, 30 Apr 2025 06:44:23 GMT"} })
145512025-04-30T06:44:23.563ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0
145522025-04-30T06:44:23.585ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "f7ccaa9c-7427-4965-96dc-ceace4334c61", "content-length": "723", "date": "Wed, 30 Apr 2025 06:44:23 GMT"} })
145532025-04-30T06:44:23.585ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0/component/sp/caboose?firmware_slot=0
145542025-04-30T06:44:23.605ZDEBGtest_utilization: attempt to fetch latest inventory collection result = Ok("found")
145552025-04-30T06:44:23.636ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "8abc8f86-7dd8-4af4-ba67-2d1b91b0cfa5", "content-length": "109", "date": "Wed, 30 Apr 2025 06:44:23 GMT"} })
145562025-04-30T06:44:23.637ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0/component/sp/caboose?firmware_slot=1
145572025-04-30T06:44:23.641ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "59139711-3dfc-42f0-a3e4-ff5be623349b", "content-length": "109", "date": "Wed, 30 Apr 2025 06:44:23 GMT"} })
145582025-04-30T06:44:23.642ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0/component/rot/caboose?firmware_slot=0
145592025-04-30T06:44:23.643ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "e34585fe-177b-4a56-978b-20e11222347d", "content-length": "169", "date": "Wed, 30 Apr 2025 06:44:23 GMT"} })
145602025-04-30T06:44:23.644ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0/component/rot/caboose?firmware_slot=1
145612025-04-30T06:44:23.646ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "d952f2f5-4e86-460f-b88c-9f70a7cefc3c", "content-length": "169", "date": "Wed, 30 Apr 2025 06:44:23 GMT"} })
145622025-04-30T06:44:23.646ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0/component/stage0/caboose?firmware_slot=0
145632025-04-30T06:44:23.650ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "892efd10-f52c-47db-bb51-5f56218202fc", "content-length": "178", "date": "Wed, 30 Apr 2025 06:44:23 GMT"} })
145642025-04-30T06:44:23.650ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0/component/stage0/caboose?firmware_slot=1
145652025-04-30T06:44:23.652ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "abded370-7ab1-4544-aac9-469b036782dd", "content-length": "178", "date": "Wed, 30 Apr 2025 06:44:23 GMT"} })
145662025-04-30T06:44:23.652ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0/component/rot/cmpa
145672025-04-30T06:44:23.653ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "52cd1a7a-be3b-4044-a7ed-caea7c6708ab", "content-length": "702", "date": "Wed, 30 Apr 2025 06:44:23 GMT"} })
145682025-04-30T06:44:23.653ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0/component/rot/cfpa
145692025-04-30T06:44:23.654ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "323f074b-3e0a-4694-bde2-7f5983d784a9", "content-length": "727", "date": "Wed, 30 Apr 2025 06:44:23 GMT"} })
145702025-04-30T06:44:23.654ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0/component/rot/cfpa
145712025-04-30T06:44:23.655ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "ba55af53-2e50-40a6-b652-e442bec04650", "content-length": "729", "date": "Wed, 30 Apr 2025 06:44:23 GMT"} })
145722025-04-30T06:44:23.655ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0/component/rot/cfpa
145732025-04-30T06:44:23.656ZTRACtest_utilization (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory
145742025-04-30T06:44:23.656ZTRACtest_utilization (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
145752025-04-30T06:44:23.656ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
145762025-04-30T06:44:23.656ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
145772025-04-30T06:44:23.657ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
145782025-04-30T06:44:23.657ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "63a8bba7-84be-4c60-89ab-2ce59c3b78d0", "content-length": "728", "date": "Wed, 30 Apr 2025 06:44:23 GMT"} })
145792025-04-30T06:44:23.657ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1
145802025-04-30T06:44:23.660ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1", status: 200, headers: {"content-type": "application/json", "x-request-id": "67c57d84-bc74-4c31-aae0-221c7b617141", "content-length": "723", "date": "Wed, 30 Apr 2025 06:44:23 GMT"} })
145812025-04-30T06:44:23.660ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1/component/sp/caboose?firmware_slot=0
145822025-04-30T06:44:23.662ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
145832025-04-30T06:44:23.669ZDEBGtest_utilization (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory result = Ok(())
145842025-04-30T06:44:23.669ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
145852025-04-30T06:44:23.669ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
145862025-04-30T06:44:23.672ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
145872025-04-30T06:44:23.676ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
145882025-04-30T06:44:23.676ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
145892025-04-30T06:44:23.676ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
145902025-04-30T06:44:23.679ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "fa91de2c-13a5-42e5-9d46-cd6743f0983e", "content-length": "109", "date": "Wed, 30 Apr 2025 06:44:23 GMT"} })
145912025-04-30T06:44:23.680ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1/component/sp/caboose?firmware_slot=1
145922025-04-30T06:44:23.690ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "e5b8ca13-a2d7-4afd-b7d0-98aafd4d9be6", "content-length": "109", "date": "Wed, 30 Apr 2025 06:44:23 GMT"} })
145932025-04-30T06:44:23.700ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1/component/rot/caboose?firmware_slot=0
145942025-04-30T06:44:23.720ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "07d081ed-b1a7-4b42-abac-ac2a1de76a14", "content-length": "169", "date": "Wed, 30 Apr 2025 06:44:23 GMT"} })
145952025-04-30T06:44:23.720ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1/component/rot/caboose?firmware_slot=1
145962025-04-30T06:44:23.723ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "0e808a41-eba5-4ca9-995a-67ba91fcdfaf", "content-length": "169", "date": "Wed, 30 Apr 2025 06:44:23 GMT"} })
145972025-04-30T06:44:23.724ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1/component/stage0/caboose?firmware_slot=0
145982025-04-30T06:44:23.728ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "0becee3c-fcd1-4089-acf9-c87db1ecb1d9", "content-length": "178", "date": "Wed, 30 Apr 2025 06:44:23 GMT"} })
145992025-04-30T06:44:23.728ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1/component/stage0/caboose?firmware_slot=1
146002025-04-30T06:44:23.732ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "4789f8ff-f2fe-4c14-bc44-62cf24487416", "content-length": "178", "date": "Wed, 30 Apr 2025 06:44:23 GMT"} })
146012025-04-30T06:44:23.732ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1/component/rot/cmpa
146022025-04-30T06:44:23.741ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "cad3d8de-f85a-4747-96cd-222f51260245", "content-length": "702", "date": "Wed, 30 Apr 2025 06:44:23 GMT"} })
146032025-04-30T06:44:23.741ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1/component/rot/cfpa
146042025-04-30T06:44:23.742ZDEBGtest_utilization: attempt to fetch latest inventory collection result = Ok("found")
146052025-04-30T06:44:23.744ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "37c7cdef-9ab7-4b1a-bd58-5b54171d2c5a", "content-length": "727", "date": "Wed, 30 Apr 2025 06:44:23 GMT"} })
146062025-04-30T06:44:23.745ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1/component/rot/cfpa
146072025-04-30T06:44:23.750ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "59ac8692-bd44-4743-af4a-18eca26e40de", "content-length": "729", "date": "Wed, 30 Apr 2025 06:44:23 GMT"} })
146082025-04-30T06:44:23.751ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1/component/rot/cfpa
146092025-04-30T06:44:23.753ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "55fb4afc-ec9d-4384-84b9-aa237cbf688f", "content-length": "728", "date": "Wed, 30 Apr 2025 06:44:23 GMT"} })
146102025-04-30T06:44:23.753ZTRACtest_utilization (BackgroundTasks): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet
146112025-04-30T06:44:23.753ZTRACtest_utilization (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
146122025-04-30T06:44:23.753ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
146132025-04-30T06:44:23.753ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
146142025-04-30T06:44:23.754ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
146152025-04-30T06:44:23.760ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
146162025-04-30T06:44:23.766ZDEBGtest_utilization (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet result = Ok(())
146172025-04-30T06:44:23.766ZTRACtest_utilization (BackgroundTasks): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet
146182025-04-30T06:44:23.766ZTRACtest_utilization (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
146192025-04-30T06:44:23.766ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
146202025-04-30T06:44:23.766ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
146212025-04-30T06:44:23.766ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
146222025-04-30T06:44:23.776ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
146232025-04-30T06:44:23.785ZDEBGtest_utilization (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet result = Ok(())
146242025-04-30T06:44:23.785ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
146252025-04-30T06:44:23.785ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
146262025-04-30T06:44:23.786ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
146272025-04-30T06:44:23.881ZDEBGtest_utilization (BackgroundTasks): begin collection from Sled Agent background_task = inventory_collection sled_agent_url = http://[::1]:39365
146282025-04-30T06:44:23.881ZDEBGtest_utilization (BackgroundTasks): client request SledAgent = http://[::1]:39365 background_task = inventory_collection body = None method = GET uri = http://[::1]:39365/inventory
146292025-04-30T06:44:23.882ZTRACtest_utilization (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory
146302025-04-30T06:44:23.882ZTRACtest_utilization (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
146312025-04-30T06:44:23.882ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
146322025-04-30T06:44:23.882ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
146332025-04-30T06:44:23.883ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
146342025-04-30T06:44:23.884ZINFOtest_utilization (dropshot): accepted connection local_addr = [::1]:39365 remote_addr = [::1]:52880 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
146352025-04-30T06:44:23.894ZTRACtest_utilization (dropshot): incoming request local_addr = [::1]:39365 method = GET remote_addr = [::1]:52880 req_id = dc172743-5935-4ffb-835f-e29d9ac8c39c sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /inventory
146362025-04-30T06:44:23.894ZINFOtest_utilization (dropshot): request completed latency_us = 447 local_addr = [::1]:39365 method = GET remote_addr = [::1]:52880 req_id = dc172743-5935-4ffb-835f-e29d9ac8c39c response_code = 200 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /inventory
146372025-04-30T06:44:23.894ZDEBGtest_utilization (BackgroundTasks): client response SledAgent = http://[::1]:39365 background_task = inventory_collection result = Ok(Response { url: "http://[::1]:39365/inventory", status: 200, headers: {"content-type": "application/json", "x-request-id": "dc172743-5935-4ffb-835f-e29d9ac8c39c", "content-length": "3141", "date": "Wed, 30 Apr 2025 06:44:23 GMT"} })
146382025-04-30T06:44:23.907ZDEBGtest_utilization (BackgroundTasks): begin collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
146392025-04-30T06:44:23.907ZDEBGtest_utilization (BackgroundTasks): end collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
146402025-04-30T06:44:23.907ZDEBGtest_utilization (BackgroundTasks): finished collection background_task = inventory_collection
146412025-04-30T06:44:23.910ZTRACtest_utilization (BackgroundTasks): authorize begin action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory
146422025-04-30T06:44:23.910ZTRACtest_utilization (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
146432025-04-30T06:44:23.910ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
146442025-04-30T06:44:23.910ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
146452025-04-30T06:44:23.913ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
146462025-04-30T06:44:23.914ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
146472025-04-30T06:44:23.923ZDEBGtest_utilization (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory result = Ok(())
146482025-04-30T06:44:23.923ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
146492025-04-30T06:44:23.923ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
146502025-04-30T06:44:23.923ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
146512025-04-30T06:44:23.926ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
146522025-04-30T06:44:23.927ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
146532025-04-30T06:44:23.927ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
146542025-04-30T06:44:23.928ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
146552025-04-30T06:44:23.931ZDEBGtest_utilization (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory result = Ok(())
146562025-04-30T06:44:23.932ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
146572025-04-30T06:44:23.932ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
146582025-04-30T06:44:23.932ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
146592025-04-30T06:44:23.958ZINFOtest_utilization (oximeter-agent): refreshing list of producers from Nexus collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 file = oximeter/collector/src/agent.rs:562
146602025-04-30T06:44:23.959ZDEBGtest_utilization (oximeter): client request body = None method = GET uri = http://127.0.0.1:32956/metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?sort_by=id_ascending
146612025-04-30T06:44:23.959ZTRACtest_utilization (dropshot_internal): incoming request local_addr = 127.0.0.1:32956 method = GET remote_addr = 127.0.0.1:55797 req_id = 3ea67011-1f0b-4e9e-ac59-ec51352ea5eb uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?sort_by=id_ascending
146622025-04-30T06:44:23.959ZTRACtest_utilization (dropshot_internal): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = 127.0.0.1:32956 method = GET remote_addr = 127.0.0.1:55797 req_id = 3ea67011-1f0b-4e9e-ac59-ec51352ea5eb resource = Database uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?sort_by=id_ascending
146632025-04-30T06:44:23.960ZDEBGtest_utilization (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = 127.0.0.1:32956 method = GET remote_addr = 127.0.0.1:55797 req_id = 3ea67011-1f0b-4e9e-ac59-ec51352ea5eb roles = RoleSet { roles: {} } uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?sort_by=id_ascending
146642025-04-30T06:44:23.961ZDEBGtest_utilization (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = 127.0.0.1:32956 method = GET remote_addr = 127.0.0.1:55797 req_id = 3ea67011-1f0b-4e9e-ac59-ec51352ea5eb resource = Database result = Ok(()) uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?sort_by=id_ascending
146652025-04-30T06:44:23.967ZINFOtest_utilization (dropshot_internal): request completed latency_us = 7674 local_addr = 127.0.0.1:32956 method = GET remote_addr = 127.0.0.1:55797 req_id = 3ea67011-1f0b-4e9e-ac59-ec51352ea5eb response_code = 200 uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?sort_by=id_ascending
146662025-04-30T06:44:23.968ZWARNtest_utilization (datastore): Non-retryable transaction failure err = DatabaseError(UniqueViolation, "duplicate key value violates unique constraint \\"inv_caboose_pkey\\"") transaction = inventory_insert_collection
146672025-04-30T06:44:23.969ZWARNtest_utilization (BackgroundTasks): inventory collection failed background_task = inventory_collection error = failed to collect inventory: saving inventory to database: Internal Error: unexpected database error: duplicate key value violates unique constraint "inv_caboose_pkey"
146682025-04-30T06:44:23.969ZDEBGtest_utilization (BackgroundTasks): activation complete background_task = inventory_collection elapsed = 908.530999ms iteration = 40 status = {"error":"failed to collect inventory: saving inventory to database: Internal Error: unexpected database error: duplicate key value violates unique constraint \\"inv_caboose_pkey\\""}
146692025-04-30T06:44:23.969ZDEBGtest_utilization (BackgroundTasks): activating background_task = inventory_collection iteration = 41 reason = Signaled
146702025-04-30T06:44:23.969ZTRACtest_utilization (BackgroundTasks): authorize begin action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory
146712025-04-30T06:44:23.969ZTRACtest_utilization (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
146722025-04-30T06:44:23.969ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
146732025-04-30T06:44:23.970ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
146742025-04-30T06:44:23.971ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
146752025-04-30T06:44:23.971ZDEBGtest_utilization (oximeter): client response result = Ok(Response { url: "http://127.0.0.1:32956/metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?sort_by=id_ascending", status: 200, headers: {"content-type": "application/json", "x-request-id": "3ea67011-1f0b-4e9e-ac59-ec51352ea5eb", "content-length": "405", "date": "Wed, 30 Apr 2025 06:44:23 GMT"} })
146762025-04-30T06:44:23.972ZDEBGtest_utilization (oximeter): client request body = None method = GET uri = http://127.0.0.1:32956/metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJhYzU1ZDhkOS1hMjk4LTQ1MzQtODk4Yi1iZWVlMWM5ZTRlNTIifX0%3D
146772025-04-30T06:44:23.972ZTRACtest_utilization (dropshot_internal): incoming request local_addr = 127.0.0.1:32956 method = GET remote_addr = 127.0.0.1:55797 req_id = 6ef0d4bf-825d-4f52-a54c-93f0187ba755 uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJhYzU1ZDhkOS1hMjk4LTQ1MzQtODk4Yi1iZWVlMWM5ZTRlNTIifX0%3D
146782025-04-30T06:44:23.973ZTRACtest_utilization (dropshot_internal): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = 127.0.0.1:32956 method = GET remote_addr = 127.0.0.1:55797 req_id = 6ef0d4bf-825d-4f52-a54c-93f0187ba755 resource = Database uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJhYzU1ZDhkOS1hMjk4LTQ1MzQtODk4Yi1iZWVlMWM5ZTRlNTIifX0%3D
146792025-04-30T06:44:23.973ZDEBGtest_utilization (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = 127.0.0.1:32956 method = GET remote_addr = 127.0.0.1:55797 req_id = 6ef0d4bf-825d-4f52-a54c-93f0187ba755 roles = RoleSet { roles: {} } uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJhYzU1ZDhkOS1hMjk4LTQ1MzQtODk4Yi1iZWVlMWM5ZTRlNTIifX0%3D
146802025-04-30T06:44:23.974ZDEBGtest_utilization (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = 127.0.0.1:32956 method = GET remote_addr = 127.0.0.1:55797 req_id = 6ef0d4bf-825d-4f52-a54c-93f0187ba755 resource = Database result = Ok(()) uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJhYzU1ZDhkOS1hMjk4LTQ1MzQtODk4Yi1iZWVlMWM5ZTRlNTIifX0%3D
146812025-04-30T06:44:23.974ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
146822025-04-30T06:44:23.981ZDEBGtest_utilization (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory result = Ok(())
146832025-04-30T06:44:23.981ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
146842025-04-30T06:44:23.981ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
146852025-04-30T06:44:23.981ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
146862025-04-30T06:44:23.985ZINFOtest_utilization (dropshot_internal): request completed latency_us = 12393 local_addr = 127.0.0.1:32956 method = GET remote_addr = 127.0.0.1:55797 req_id = 6ef0d4bf-825d-4f52-a54c-93f0187ba755 response_code = 200 uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJhYzU1ZDhkOS1hMjk4LTQ1MzQtODk4Yi1iZWVlMWM5ZTRlNTIifX0%3D
146872025-04-30T06:44:23.985ZDEBGtest_utilization: attempt to fetch latest inventory collection result = Ok("found")
146882025-04-30T06:44:23.986ZDEBGtest_utilization (oximeter): client response result = Ok(Response { url: "http://127.0.0.1:32956/metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJhYzU1ZDhkOS1hMjk4LTQ1MzQtODk4Yi1iZWVlMWM5ZTRlNTIifX0%3D", status: 200, headers: {"content-type": "application/json", "x-request-id": "6ef0d4bf-825d-4f52-a54c-93f0187ba755", "content-length": "29", "date": "Wed, 30 Apr 2025 06:44:23 GMT"} })
146892025-04-30T06:44:23.986ZTRACtest_utilization (oximeter-agent): ignoring request to update existing metric producer, since the endpoint information is the same as the existing collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
146902025-04-30T06:44:23.986ZTRACtest_utilization (oximeter-agent): ignoring request to update existing metric producer, since the endpoint information is the same as the existing collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = ac55d8d9-a298-4534-898b-beee1c9e4e52
146912025-04-30T06:44:23.986ZINFOtest_utilization (oximeter-agent): refreshed list of producers from Nexus collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 file = oximeter/collector/src/agent.rs:637 n_current_tasks = 2 n_pruned_tasks = 0
146922025-04-30T06:44:23.987ZDEBGtest_utilization (BackgroundTasks): inventory_prune_one: nothing eligible for removal (too few) background_task = inventory_collection candidates = [(13d8b8d9-6c6b-47a6-9190-952544edb75b, 0)]
146932025-04-30T06:44:23.987ZTRACtest_utilization (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal
146942025-04-30T06:44:23.988ZTRACtest_utilization (dns): buffer buffer = Length: 67 (0x43) bytes\n0000: 87 17 01 00 00 01 00 00 00 00 00 01 04 5f 6d 67 ............._mg\n0010: 73 04 5f 74 63 70 0d 63 6f 6e 74 72 6f 6c 2d 70 s._tcp.control-p\n0020: 6c 61 6e 65 05 6f 78 69 64 65 08 69 6e 74 65 72 lane.oxide.inter\n0030: 6e 61 6c 00 00 21 00 01 00 00 29 04 d0 00 00 00 nal..!....).....\n0040: 00 00 00 ... kind = dns peer_addr = [::1]:53603 req_id = eb252589-eae0-4dbd-abc3-c5c15d2c3e0a
146952025-04-30T06:44:23.988ZDEBGtest_utilization (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 34583,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_mgs._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_mgs._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 4,\n 95,\n 109,\n 103,\n 115,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:53603 req_id = eb252589-eae0-4dbd-abc3-c5c15d2c3e0a
146962025-04-30T06:44:23.988ZDEBGtest_utilization (store): query key key = _mgs._tcp
146972025-04-30T06:44:23.988ZDEBGtest_utilization (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
146982025-04-30T06:44:23.988ZDEBGtest_utilization (dns): dns response additional_records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal"), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] kind = dns peer_addr = [::1]:53603 query = LowerQuery { name: LowerName(Name("_mgs._tcp.control-plane.oxide.internal.")), original: Query { name: Name("_mgs._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN } } records = [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 48759, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal") })) }] req_id = eb252589-eae0-4dbd-abc3-c5c15d2c3e0a
146992025-04-30T06:44:23.989ZDEBGtest_utilization (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgs._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 48759, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }], valid_until: Instant { tv_sec: 4328, tv_nsec: 788360851 } })
147002025-04-30T06:44:23.989ZTRACtest_utilization (DnsResolver): lookup_service_targets: looking up SRV target
147012025-04-30T06:44:23.989ZTRACtest_utilization (dns): buffer buffer = Length: 108 (0x6c) bytes\n0000: 0e ee 01 00 00 01 00 00 00 00 00 01 2d 64 65 6e ............-den\n0010: 64 72 69 74 65 2d 62 36 64 36 35 33 34 31 2d 31 drite-b6d65341-1\n0020: 36 37 63 2d 34 31 64 66 2d 39 62 35 63 2d 34 31 67c-41df-9b5c-41\n0030: 63 64 65 64 39 39 63 32 32 39 04 68 6f 73 74 0d cded99c229.host.\n0040: 63 6f 6e 74 72 6f 6c 2d 70 6c 61 6e 65 05 6f 78 control-plane.ox\n0050: 69 64 65 08 69 6e 74 65 72 6e 61 6c 00 00 1c 00 ide.internal....\n0060: 01 00 00 29 04 d0 00 00 00 00 00 00 ...)........ kind = dns peer_addr = [::1]:64941 req_id = 679b1cee-206d-4e92-af09-c0e1068aff43
147022025-04-30T06:44:23.989ZDEBGtest_utilization (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 3822,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n query_type: AAAA,\n query_class: IN,\n },\n },\n original: [\n 45,\n 100,\n 101,\n 110,\n 100,\n 114,\n 105,\n 116,\n 101,\n 45,\n 98,\n 54,\n 100,\n 54,\n 53,\n 51,\n 52,\n 49,\n 45,\n 49,\n 54,\n 55,\n 99,\n 45,\n 52,\n 49,\n 100,\n 102,\n 45,\n 57,\n 98,\n 53,\n 99,\n 45,\n 52,\n 49,\n 99,\n 100,\n 101,\n 100,\n 57,\n 57,\n 99,\n 50,\n 50,\n 57,\n 4,\n 104,\n 111,\n 115,\n 116,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 28,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:64941 req_id = 679b1cee-206d-4e92-af09-c0e1068aff43
147032025-04-30T06:44:23.990ZDEBGtest_utilization (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
147042025-04-30T06:44:23.990ZDEBGtest_utilization (dns): dns response additional_records = [] kind = dns peer_addr = [::1]:64941 query = LowerQuery { name: LowerName(Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.")), original: Query { name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), query_type: AAAA, query_class: IN } } records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] req_id = 679b1cee-206d-4e92-af09-c0e1068aff43
147052025-04-30T06:44:24.045ZTRACtest_utilization (DnsResolver): lookup_all_socket_v6 srv dns_name = _clickhouse-admin-keeper._tcp.control-plane.oxide.internal
147062025-04-30T06:44:24.045ZTRACtest_utilization (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory
147072025-04-30T06:44:24.045ZTRACtest_utilization (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
147082025-04-30T06:44:24.046ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
147092025-04-30T06:44:24.046ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
147102025-04-30T06:44:24.047ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
147112025-04-30T06:44:24.048ZDEBGtest_utilization (collection-task): sent timer-based collection request to the collection task collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
147122025-04-30T06:44:24.049ZDEBGtest_utilization (collection-task): collecting from producer collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
147132025-04-30T06:44:24.050ZTRACtest_utilization (dns): buffer buffer = Length: 87 (0x57) bytes\n0000: c3 2a 01 00 00 01 00 00 00 00 00 01 18 5f 63 6c .*..........._cl\n0010: 69 63 6b 68 6f 75 73 65 2d 61 64 6d 69 6e 2d 6b ickhouse-admin-k\n0020: 65 65 70 65 72 04 5f 74 63 70 0d 63 6f 6e 74 72 eeper._tcp.contr\n0030: 6f 6c 2d 70 6c 61 6e 65 05 6f 78 69 64 65 08 69 ol-plane.oxide.i\n0040: 6e 74 65 72 6e 61 6c 00 00 21 00 01 00 00 29 04 nternal..!....).\n0050: d0 00 00 00 00 00 00 ....... kind = dns peer_addr = [::1]:62209 req_id = 91264585-60fe-4b5c-9068-7ac8a55adbe1
147142025-04-30T06:44:24.050ZDEBGtest_utilization (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 49962,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 24,\n 95,\n 99,\n 108,\n 105,\n 99,\n 107,\n 104,\n 111,\n 117,\n 115,\n 101,\n 45,\n 97,\n 100,\n 109,\n 105,\n 110,\n 45,\n 107,\n 101,\n 101,\n 112,\n 101,\n 114,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:62209 req_id = 91264585-60fe-4b5c-9068-7ac8a55adbe1
147152025-04-30T06:44:24.050ZDEBGtest_utilization (store): query key key = _clickhouse-admin-keeper._tcp
147162025-04-30T06:44:24.050ZERROtest_utilization (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 49962, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 1, }, query: WireQuery { query: LowerQuery { name: LowerName( Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), ), original: Query { name: Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN, }, }, original: [ 24, 95, 99, 108, 105, 99, 107, 104, 111, 117, 115, 101, 45, 97, 100, 109, 105, 110, 45, 107, 101, 101, 112, 101, 114, 4, 95, 116, 99, 112, 13, 99, 111, 110, 116, 114, 111, 108, 45, 112, 108, 97, 110, 101, 5, 111, 120, 105, 100, 101, 8, 105, 110, 116, 101, 114, 110, 97, 108, 0, 0, 33, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: Some( Edns { rcode_high: 0, version: 0, dnssec_ok: false, max_payload: 1232, options: OPT { options: {}, }, }, ), } NXDOMAIN: no records found for name: "_clickhouse-admin-keeper._tcp.control-plane.oxide.internal." kind = dns peer_addr = [::1]:62209 req_id = 91264585-60fe-4b5c-9068-7ac8a55adbe1
147172025-04-30T06:44:24.052ZDEBGtest_utilization (BackgroundTasks): begin collection background_task = inventory_collection
147182025-04-30T06:44:24.052ZDEBGtest_utilization (BackgroundTasks): begin collection from MGS background_task = inventory_collection mgs_url = http://[::1]:48759
147192025-04-30T06:44:24.052ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/ignition
147202025-04-30T06:44:24.053ZTRACtest_utilization (collection-task): sent collection request to producer collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
147212025-04-30T06:44:24.054ZDEBGtest_utilization (collection-task): collected results from producer collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 n_results = 1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
147222025-04-30T06:44:24.054ZTRACtest_utilization (collection-task): forwarded results to main collection loop collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
147232025-04-30T06:44:24.054ZTRACtest_utilization (collection-task): top of inner collection loop, waiting for next request collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
147242025-04-30T06:44:24.054ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
147252025-04-30T06:44:24.059ZDEBGtest_utilization (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory result = Ok(())
147262025-04-30T06:44:24.059ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
147272025-04-30T06:44:24.059ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
147282025-04-30T06:44:24.060ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
147292025-04-30T06:44:24.064ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
147302025-04-30T06:44:24.064ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
147312025-04-30T06:44:24.064ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
147322025-04-30T06:44:24.071ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/ignition", status: 200, headers: {"content-type": "application/json", "x-request-id": "90630d89-d9c5-4c73-b0b9-f961c55f4019", "content-length": "835", "date": "Wed, 30 Apr 2025 06:44:23 GMT"} })
147332025-04-30T06:44:24.071ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0
147342025-04-30T06:44:24.074ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "c19a45e0-d0d4-4b16-b0b0-4faf89784ddb", "content-length": "734", "date": "Wed, 30 Apr 2025 06:44:23 GMT"} })
147352025-04-30T06:44:24.075ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0/component/sp/caboose?firmware_slot=0
147362025-04-30T06:44:24.084ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "29af3988-1460-43d3-a91b-d820f1306d1c", "content-length": "111", "date": "Wed, 30 Apr 2025 06:44:23 GMT"} })
147372025-04-30T06:44:24.084ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0/component/sp/caboose?firmware_slot=1
147382025-04-30T06:44:24.087ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "ac0da542-8bfb-4193-939c-a8a96edcb41b", "content-length": "111", "date": "Wed, 30 Apr 2025 06:44:23 GMT"} })
147392025-04-30T06:44:24.087ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0/component/rot/caboose?firmware_slot=0
147402025-04-30T06:44:24.089ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "9e1f4fa4-97f2-41ab-9c8d-c0921c4301ed", "content-length": "170", "date": "Wed, 30 Apr 2025 06:44:23 GMT"} })
147412025-04-30T06:44:24.090ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0/component/rot/caboose?firmware_slot=1
147422025-04-30T06:44:24.091ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "e74da672-e311-449f-b638-109923c554a2", "content-length": "170", "date": "Wed, 30 Apr 2025 06:44:23 GMT"} })
147432025-04-30T06:44:24.091ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0/component/stage0/caboose?firmware_slot=0
147442025-04-30T06:44:24.104ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "7d9f6dd2-e169-443a-bea9-6919753401f4", "content-length": "179", "date": "Wed, 30 Apr 2025 06:44:23 GMT"} })
147452025-04-30T06:44:24.104ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0/component/stage0/caboose?firmware_slot=1
147462025-04-30T06:44:24.109ZDEBGtest_utilization: attempt to fetch latest inventory collection result = Ok("found")
147472025-04-30T06:44:24.110ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "c97362c0-5bae-46fe-b63b-a939a24e4cc8", "content-length": "179", "date": "Wed, 30 Apr 2025 06:44:23 GMT"} })
147482025-04-30T06:44:24.110ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0/component/rot/cmpa
147492025-04-30T06:44:24.112ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "53649bc5-904e-4bc3-818c-2dfa029b9399", "content-length": "702", "date": "Wed, 30 Apr 2025 06:44:23 GMT"} })
147502025-04-30T06:44:24.112ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0/component/rot/cfpa
147512025-04-30T06:44:24.115ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "97ac2966-643e-4c0a-952c-e78c13aedb71", "content-length": "727", "date": "Wed, 30 Apr 2025 06:44:23 GMT"} })
147522025-04-30T06:44:24.116ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0/component/rot/cfpa
147532025-04-30T06:44:24.117ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "77e0dc13-9d9a-4b50-bf90-f26ef2a81203", "content-length": "729", "date": "Wed, 30 Apr 2025 06:44:23 GMT"} })
147542025-04-30T06:44:24.117ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0/component/rot/cfpa
147552025-04-30T06:44:24.119ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "062f3412-8e72-4c7b-a1df-1e18a665f7d2", "content-length": "728", "date": "Wed, 30 Apr 2025 06:44:23 GMT"} })
147562025-04-30T06:44:24.119ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1
147572025-04-30T06:44:24.122ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1", status: 200, headers: {"content-type": "application/json", "x-request-id": "59249657-2813-4145-9df9-b59dbb881069", "content-length": "734", "date": "Wed, 30 Apr 2025 06:44:23 GMT"} })
147582025-04-30T06:44:24.123ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1/component/sp/caboose?firmware_slot=0
147592025-04-30T06:44:24.128ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "2935a201-fc01-4ceb-b0fa-353a17c2ee9b", "content-length": "111", "date": "Wed, 30 Apr 2025 06:44:23 GMT"} })
147602025-04-30T06:44:24.128ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1/component/sp/caboose?firmware_slot=1
147612025-04-30T06:44:24.130ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "436a65db-a659-4c17-86c6-976cf68a78be", "content-length": "111", "date": "Wed, 30 Apr 2025 06:44:23 GMT"} })
147622025-04-30T06:44:24.131ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1/component/rot/caboose?firmware_slot=0
147632025-04-30T06:44:24.133ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "3b80ff8d-728b-43bc-b2fc-7452fb82ecbe", "content-length": "170", "date": "Wed, 30 Apr 2025 06:44:23 GMT"} })
147642025-04-30T06:44:24.134ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1/component/rot/caboose?firmware_slot=1
147652025-04-30T06:44:24.138ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "1e2e9040-4893-476b-bbb1-d93f3d35e2b1", "content-length": "170", "date": "Wed, 30 Apr 2025 06:44:23 GMT"} })
147662025-04-30T06:44:24.138ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1/component/stage0/caboose?firmware_slot=0
147672025-04-30T06:44:24.139ZDEBGtest_utilization (collection-task): sent timer-based collection request to the collection task collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = ac55d8d9-a298-4534-898b-beee1c9e4e52
147682025-04-30T06:44:24.139ZDEBGtest_utilization (collection-task): collecting from producer collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = ac55d8d9-a298-4534-898b-beee1c9e4e52
147692025-04-30T06:44:24.140ZTRACtest_utilization (dropshot): incoming request local_addr = 127.0.0.1:57688 method = GET remote_addr = 127.0.0.1:55243 req_id = 7450d1c6-b703-42c7-bbf1-48a09c5e7611 uri = /ac55d8d9-a298-4534-898b-beee1c9e4e52
147702025-04-30T06:44:24.159ZINFOtest_utilization (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 18421 local_addr = 127.0.0.1:57688 method = GET remote_addr = 127.0.0.1:55243 req_id = 7450d1c6-b703-42c7-bbf1-48a09c5e7611 response_code = 200 uri = /ac55d8d9-a298-4534-898b-beee1c9e4e52
147712025-04-30T06:44:24.159ZTRACtest_utilization (collection-task): sent collection request to producer collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = ac55d8d9-a298-4534-898b-beee1c9e4e52
147722025-04-30T06:44:24.186ZDEBGtest_utilization (collection-task): collected results from producer collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 n_results = 5 producer_id = ac55d8d9-a298-4534-898b-beee1c9e4e52
147732025-04-30T06:44:24.186ZTRACtest_utilization (collection-task): forwarded results to main collection loop collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = ac55d8d9-a298-4534-898b-beee1c9e4e52
147742025-04-30T06:44:24.186ZTRACtest_utilization (collection-task): top of inner collection loop, waiting for next request collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = ac55d8d9-a298-4534-898b-beee1c9e4e52
147752025-04-30T06:44:24.187ZDEBGtest_utilization (results-sink): inserting 16 samples into database collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
147762025-04-30T06:44:24.187ZDEBGtest_utilization (results-sink-cluster): inserting 16 samples into database collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
147772025-04-30T06:44:24.187ZDEBGtest_utilization (clickhouse-client): unrolling 16 total samples collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 4d1485f0-110e-4161-9a33-c4610d68be5d
147782025-04-30T06:44:24.204ZTRACtest_utilization (clickhouse-client): inserting data collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 4d1485f0-110e-4161-9a33-c4610d68be5d n_columns = 4 n_rows = 1 sql = INSERT INTO oximeter.fields_ipaddr FORMAT Native
147792025-04-30T06:44:24.206ZTRACtest_utilization (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory
147802025-04-30T06:44:24.206ZTRACtest_utilization (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
147812025-04-30T06:44:24.206ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
147822025-04-30T06:44:24.206ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
147832025-04-30T06:44:24.206ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
147842025-04-30T06:44:24.210ZDEBGtest_utilization (clickhouse-client): inserted rows into table collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 4d1485f0-110e-4161-9a33-c4610d68be5d n_rows = 1 table_name = fields_ipaddr
147852025-04-30T06:44:24.210ZTRACtest_utilization (clickhouse-client): inserting data collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 4d1485f0-110e-4161-9a33-c4610d68be5d n_columns = 4 n_rows = 32 sql = INSERT INTO oximeter.fields_string FORMAT Native
147862025-04-30T06:44:24.215ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
147872025-04-30T06:44:24.221ZDEBGtest_utilization (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory result = Ok(())
147882025-04-30T06:44:24.221ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
147892025-04-30T06:44:24.221ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
147902025-04-30T06:44:24.221ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
147912025-04-30T06:44:24.221ZDEBGtest_utilization (clickhouse-client): inserted rows into table collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 4d1485f0-110e-4161-9a33-c4610d68be5d n_rows = 32 table_name = fields_string
147922025-04-30T06:44:24.222ZTRACtest_utilization (clickhouse-client): inserting data collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 4d1485f0-110e-4161-9a33-c4610d68be5d n_columns = 4 n_rows = 15 sql = INSERT INTO oximeter.fields_u16 FORMAT Native
147932025-04-30T06:44:24.223ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "2e7a3bef-8414-4701-8db2-9a8207c02cd6", "content-length": "179", "date": "Wed, 30 Apr 2025 06:44:24 GMT"} })
147942025-04-30T06:44:24.223ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1/component/stage0/caboose?firmware_slot=1
147952025-04-30T06:44:24.228ZDEBGtest_utilization (clickhouse-client): inserted rows into table collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 4d1485f0-110e-4161-9a33-c4610d68be5d n_rows = 15 table_name = fields_u16
147962025-04-30T06:44:24.228ZTRACtest_utilization (clickhouse-client): inserting data collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 4d1485f0-110e-4161-9a33-c4610d68be5d n_columns = 4 n_rows = 21 sql = INSERT INTO oximeter.fields_uuid FORMAT Native
147972025-04-30T06:44:24.229ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
147982025-04-30T06:44:24.229ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
147992025-04-30T06:44:24.229ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
148002025-04-30T06:44:24.233ZDEBGtest_utilization (clickhouse-client): inserted rows into table collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 4d1485f0-110e-4161-9a33-c4610d68be5d n_rows = 21 table_name = fields_uuid
148012025-04-30T06:44:24.233ZTRACtest_utilization (clickhouse-client): inserting data collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 4d1485f0-110e-4161-9a33-c4610d68be5d n_columns = 5 n_rows = 1 sql = INSERT INTO oximeter.measurements_cumulativeu64 FORMAT Native
148022025-04-30T06:44:24.237ZDEBGtest_utilization (clickhouse-client): inserted rows into table collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 4d1485f0-110e-4161-9a33-c4610d68be5d n_rows = 1 table_name = measurements_cumulativeu64
148032025-04-30T06:44:24.237ZTRACtest_utilization (clickhouse-client): inserting data collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 4d1485f0-110e-4161-9a33-c4610d68be5d n_columns = 19 n_rows = 14 sql = INSERT INTO oximeter.measurements_histogramu64 FORMAT Native
148042025-04-30T06:44:24.244ZDEBGtest_utilization (clickhouse-client): inserted rows into table collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 4d1485f0-110e-4161-9a33-c4610d68be5d n_rows = 14 table_name = measurements_histogramu64
148052025-04-30T06:44:24.244ZTRACtest_utilization (clickhouse-client): inserting data collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 4d1485f0-110e-4161-9a33-c4610d68be5d n_columns = 4 n_rows = 1 sql = INSERT INTO oximeter.measurements_i64 FORMAT Native
148062025-04-30T06:44:24.248ZDEBGtest_utilization (clickhouse-client): inserted rows into table collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 4d1485f0-110e-4161-9a33-c4610d68be5d n_rows = 1 table_name = measurements_i64
148072025-04-30T06:44:24.249ZTRACtest_utilization (results-sink): successfully inserted samples collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
148082025-04-30T06:44:24.251ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "e41c4a0f-7e1e-49a9-8b8b-c5bba1d5bd2c", "content-length": "179", "date": "Wed, 30 Apr 2025 06:44:24 GMT"} })
148092025-04-30T06:44:24.252ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1/component/rot/cmpa
148102025-04-30T06:44:24.258ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "0b795d3f-6bfd-4602-ac05-9a50a58ce87d", "content-length": "702", "date": "Wed, 30 Apr 2025 06:44:24 GMT"} })
148112025-04-30T06:44:24.258ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1/component/rot/cfpa
148122025-04-30T06:44:24.265ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "07618a86-4683-4ed6-b339-cae0079a92ef", "content-length": "727", "date": "Wed, 30 Apr 2025 06:44:24 GMT"} })
148132025-04-30T06:44:24.267ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1/component/rot/cfpa
148142025-04-30T06:44:24.269ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "3b7d6652-948c-480c-8872-05148fbbd921", "content-length": "729", "date": "Wed, 30 Apr 2025 06:44:24 GMT"} })
148152025-04-30T06:44:24.269ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1/component/rot/cfpa
148162025-04-30T06:44:24.271ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "31d0d68c-db9d-4747-a01d-2b308af15184", "content-length": "728", "date": "Wed, 30 Apr 2025 06:44:24 GMT"} })
148172025-04-30T06:44:24.271ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0
148182025-04-30T06:44:24.274ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "3eeae863-1e26-4d52-a6c1-0972b944da3f", "content-length": "723", "date": "Wed, 30 Apr 2025 06:44:24 GMT"} })
148192025-04-30T06:44:24.274ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0/component/sp/caboose?firmware_slot=0
148202025-04-30T06:44:24.279ZDEBGtest_utilization: attempt to fetch latest inventory collection result = Ok("found")
148212025-04-30T06:44:24.283ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "e9125a14-37bf-48f0-a4da-2c479ca84110", "content-length": "109", "date": "Wed, 30 Apr 2025 06:44:24 GMT"} })
148222025-04-30T06:44:24.284ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0/component/sp/caboose?firmware_slot=1
148232025-04-30T06:44:24.289ZWARNtest_utilization (results-sink-cluster): failed to insert some results into metric DB collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 error = Failed to check out connection to database: No backends found for this service file = oximeter/collector/src/results_sink.rs:92
148242025-04-30T06:44:24.289ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "6975803c-df85-4aab-8c9d-640735b560a0", "content-length": "109", "date": "Wed, 30 Apr 2025 06:44:24 GMT"} })
148252025-04-30T06:44:24.290ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0/component/rot/caboose?firmware_slot=0
148262025-04-30T06:44:24.294ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "34db844e-9fee-461d-b023-f4472b3e2050", "content-length": "169", "date": "Wed, 30 Apr 2025 06:44:24 GMT"} })
148272025-04-30T06:44:24.295ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0/component/rot/caboose?firmware_slot=1
148282025-04-30T06:44:24.300ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "7b0738c4-87ce-4145-8392-3159cb323081", "content-length": "169", "date": "Wed, 30 Apr 2025 06:44:24 GMT"} })
148292025-04-30T06:44:24.300ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0/component/stage0/caboose?firmware_slot=0
148302025-04-30T06:44:24.304ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "794d10a5-b537-4370-983d-e312eed8525d", "content-length": "178", "date": "Wed, 30 Apr 2025 06:44:24 GMT"} })
148312025-04-30T06:44:24.304ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0/component/stage0/caboose?firmware_slot=1
148322025-04-30T06:44:24.309ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "3c76a00e-85ef-43a7-aaac-45ed0ae2033a", "content-length": "178", "date": "Wed, 30 Apr 2025 06:44:24 GMT"} })
148332025-04-30T06:44:24.309ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0/component/rot/cmpa
148342025-04-30T06:44:24.311ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "63be56c0-31f8-4798-948e-2f7b51d179bf", "content-length": "702", "date": "Wed, 30 Apr 2025 06:44:24 GMT"} })
148352025-04-30T06:44:24.311ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0/component/rot/cfpa
148362025-04-30T06:44:24.312ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "958e992c-8ba6-41a5-90dd-614b58073b63", "content-length": "727", "date": "Wed, 30 Apr 2025 06:44:24 GMT"} })
148372025-04-30T06:44:24.313ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0/component/rot/cfpa
148382025-04-30T06:44:24.317ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "d884d8bb-b3ab-43e0-bc90-1df830ef04f6", "content-length": "729", "date": "Wed, 30 Apr 2025 06:44:24 GMT"} })
148392025-04-30T06:44:24.317ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0/component/rot/cfpa
148402025-04-30T06:44:24.322ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "e202f0d7-0945-4174-b12b-612503dfe706", "content-length": "728", "date": "Wed, 30 Apr 2025 06:44:24 GMT"} })
148412025-04-30T06:44:24.322ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1
148422025-04-30T06:44:24.338ZTRACtest_utilization (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory
148432025-04-30T06:44:24.338ZTRACtest_utilization (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
148442025-04-30T06:44:24.338ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
148452025-04-30T06:44:24.338ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
148462025-04-30T06:44:24.339ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
148472025-04-30T06:44:24.339ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1", status: 200, headers: {"content-type": "application/json", "x-request-id": "721a5226-fde1-4c2d-a814-d978dc8597fe", "content-length": "723", "date": "Wed, 30 Apr 2025 06:44:24 GMT"} })
148482025-04-30T06:44:24.340ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1/component/sp/caboose?firmware_slot=0
148492025-04-30T06:44:24.343ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
148502025-04-30T06:44:24.349ZDEBGtest_utilization (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory result = Ok(())
148512025-04-30T06:44:24.353ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
148522025-04-30T06:44:24.359ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
148532025-04-30T06:44:24.362ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
148542025-04-30T06:44:24.386ZTRACtest_utilization (results-sink): batch interval expired, but no samples to insert collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
148552025-04-30T06:44:24.387ZTRACtest_utilization (results-sink-cluster): batch interval expired, but no samples to insert collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
148562025-04-30T06:44:24.392ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
148572025-04-30T06:44:24.392ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
148582025-04-30T06:44:24.393ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
148592025-04-30T06:44:24.396ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "7c35bfc6-bf49-4c75-96c3-25546c67ed60", "content-length": "109", "date": "Wed, 30 Apr 2025 06:44:24 GMT"} })
148602025-04-30T06:44:24.397ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1/component/sp/caboose?firmware_slot=1
148612025-04-30T06:44:24.399ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "94992544-c8b0-409b-9bfc-0c402396251c", "content-length": "109", "date": "Wed, 30 Apr 2025 06:44:24 GMT"} })
148622025-04-30T06:44:24.399ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1/component/rot/caboose?firmware_slot=0
148632025-04-30T06:44:24.407ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "25383694-b770-4cf4-8618-2293078f57d4", "content-length": "169", "date": "Wed, 30 Apr 2025 06:44:24 GMT"} })
148642025-04-30T06:44:24.407ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1/component/rot/caboose?firmware_slot=1
148652025-04-30T06:44:24.409ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "5cd025e4-f84c-4c81-9fe2-650d5e77f911", "content-length": "169", "date": "Wed, 30 Apr 2025 06:44:24 GMT"} })
148662025-04-30T06:44:24.409ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1/component/stage0/caboose?firmware_slot=0
148672025-04-30T06:44:24.415ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "c7ad5291-0cc4-41d7-8acb-a6c311ad6c29", "content-length": "178", "date": "Wed, 30 Apr 2025 06:44:24 GMT"} })
148682025-04-30T06:44:24.416ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1/component/stage0/caboose?firmware_slot=1
148692025-04-30T06:44:24.427ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "109ce105-c4bf-4cd3-a4b9-57681c8394fa", "content-length": "178", "date": "Wed, 30 Apr 2025 06:44:24 GMT"} })
148702025-04-30T06:44:24.427ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1/component/rot/cmpa
148712025-04-30T06:44:24.429ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "9180fd11-4468-404e-ab7e-f76e39a37644", "content-length": "702", "date": "Wed, 30 Apr 2025 06:44:24 GMT"} })
148722025-04-30T06:44:24.430ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1/component/rot/cfpa
148732025-04-30T06:44:24.431ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "6a548abd-e94d-4521-9181-8422e64ca561", "content-length": "727", "date": "Wed, 30 Apr 2025 06:44:24 GMT"} })
148742025-04-30T06:44:24.431ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1/component/rot/cfpa
148752025-04-30T06:44:24.432ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "43240e8b-b54d-4bbd-b299-0cba29fe47a5", "content-length": "729", "date": "Wed, 30 Apr 2025 06:44:24 GMT"} })
148762025-04-30T06:44:24.433ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1/component/rot/cfpa
148772025-04-30T06:44:24.452ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "f6681a89-2362-4325-b681-be91e90a4aca", "content-length": "728", "date": "Wed, 30 Apr 2025 06:44:24 GMT"} })
148782025-04-30T06:44:24.452ZTRACtest_utilization (BackgroundTasks): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet
148792025-04-30T06:44:24.452ZTRACtest_utilization (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
148802025-04-30T06:44:24.452ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
148812025-04-30T06:44:24.452ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
148822025-04-30T06:44:24.454ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
148832025-04-30T06:44:24.455ZDEBGtest_utilization: attempt to fetch latest inventory collection result = Ok("found")
148842025-04-30T06:44:24.466ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
148852025-04-30T06:44:24.475ZDEBGtest_utilization (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet result = Ok(())
148862025-04-30T06:44:24.476ZTRACtest_utilization (BackgroundTasks): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet
148872025-04-30T06:44:24.477ZTRACtest_utilization (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
148882025-04-30T06:44:24.477ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
148892025-04-30T06:44:24.477ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
148902025-04-30T06:44:24.478ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
148912025-04-30T06:44:24.493ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
148922025-04-30T06:44:24.508ZDEBGtest_utilization (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet result = Ok(())
148932025-04-30T06:44:24.508ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
148942025-04-30T06:44:24.508ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
148952025-04-30T06:44:24.508ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
148962025-04-30T06:44:24.509ZTRACtest_utilization (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory
148972025-04-30T06:44:24.509ZTRACtest_utilization (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
148982025-04-30T06:44:24.509ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
148992025-04-30T06:44:24.509ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
149002025-04-30T06:44:24.510ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
149012025-04-30T06:44:24.612ZDEBGtest_utilization (BackgroundTasks): begin collection from Sled Agent background_task = inventory_collection sled_agent_url = http://[::1]:39365
149022025-04-30T06:44:24.612ZDEBGtest_utilization (BackgroundTasks): client request SledAgent = http://[::1]:39365 background_task = inventory_collection body = None method = GET uri = http://[::1]:39365/inventory
149032025-04-30T06:44:24.614ZINFOtest_utilization (dropshot): accepted connection local_addr = [::1]:39365 remote_addr = [::1]:42320 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
149042025-04-30T06:44:24.617ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
149052025-04-30T06:44:24.625ZDEBGtest_utilization (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory result = Ok(())
149062025-04-30T06:44:24.625ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
149072025-04-30T06:44:24.625ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
149082025-04-30T06:44:24.626ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
149092025-04-30T06:44:24.626ZTRACtest_utilization (dropshot): incoming request local_addr = [::1]:39365 method = GET remote_addr = [::1]:42320 req_id = b666db4a-ebff-4bc9-ad02-93bfc350ed53 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /inventory
149102025-04-30T06:44:24.627ZINFOtest_utilization (dropshot): request completed latency_us = 650 local_addr = [::1]:39365 method = GET remote_addr = [::1]:42320 req_id = b666db4a-ebff-4bc9-ad02-93bfc350ed53 response_code = 200 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /inventory
149112025-04-30T06:44:24.628ZDEBGtest_utilization (BackgroundTasks): client response SledAgent = http://[::1]:39365 background_task = inventory_collection result = Ok(Response { url: "http://[::1]:39365/inventory", status: 200, headers: {"content-type": "application/json", "x-request-id": "b666db4a-ebff-4bc9-ad02-93bfc350ed53", "content-length": "3141", "date": "Wed, 30 Apr 2025 06:44:24 GMT"} })
149122025-04-30T06:44:24.629ZDEBGtest_utilization (BackgroundTasks): begin collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
149132025-04-30T06:44:24.629ZDEBGtest_utilization (BackgroundTasks): end collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
149142025-04-30T06:44:24.629ZDEBGtest_utilization (BackgroundTasks): finished collection background_task = inventory_collection
149152025-04-30T06:44:24.631ZTRACtest_utilization (BackgroundTasks): authorize begin action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory
149162025-04-30T06:44:24.631ZTRACtest_utilization (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
149172025-04-30T06:44:24.631ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
149182025-04-30T06:44:24.631ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
149192025-04-30T06:44:24.632ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
149202025-04-30T06:44:24.635ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
149212025-04-30T06:44:24.635ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
149222025-04-30T06:44:24.636ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
149232025-04-30T06:44:24.640ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
149242025-04-30T06:44:24.645ZDEBGtest_utilization (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory result = Ok(())
149252025-04-30T06:44:24.645ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
149262025-04-30T06:44:24.645ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
149272025-04-30T06:44:24.645ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
149282025-04-30T06:44:24.717ZWARNtest_utilization (datastore): Non-retryable transaction failure err = DatabaseError(UniqueViolation, "duplicate key value violates unique constraint \\"inv_caboose_pkey\\"") transaction = inventory_insert_collection
149292025-04-30T06:44:24.717ZWARNtest_utilization (BackgroundTasks): inventory collection failed background_task = inventory_collection error = failed to collect inventory: saving inventory to database: Internal Error: unexpected database error: duplicate key value violates unique constraint "inv_caboose_pkey"
149302025-04-30T06:44:24.718ZDEBGtest_utilization (BackgroundTasks): activation complete background_task = inventory_collection elapsed = 748.519596ms iteration = 41 status = {"error":"failed to collect inventory: saving inventory to database: Internal Error: unexpected database error: duplicate key value violates unique constraint \\"inv_caboose_pkey\\""}
149312025-04-30T06:44:24.718ZDEBGtest_utilization (BackgroundTasks): activating background_task = inventory_collection iteration = 42 reason = Signaled
149322025-04-30T06:44:24.718ZTRACtest_utilization (BackgroundTasks): authorize begin action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory
149332025-04-30T06:44:24.718ZTRACtest_utilization (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
149342025-04-30T06:44:24.718ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
149352025-04-30T06:44:24.718ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
149362025-04-30T06:44:24.718ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
149372025-04-30T06:44:24.722ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
149382025-04-30T06:44:24.746ZDEBGtest_utilization (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory result = Ok(())
149392025-04-30T06:44:24.747ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
149402025-04-30T06:44:24.747ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
149412025-04-30T06:44:24.747ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
149422025-04-30T06:44:24.748ZDEBGtest_utilization: attempt to fetch latest inventory collection result = Ok("found")
149432025-04-30T06:44:24.766ZDEBGtest_utilization (BackgroundTasks): inventory_prune_one: nothing eligible for removal (too few) background_task = inventory_collection candidates = [(13d8b8d9-6c6b-47a6-9190-952544edb75b, 0)]
149442025-04-30T06:44:24.766ZTRACtest_utilization (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal
149452025-04-30T06:44:24.768ZTRACtest_utilization (dns): buffer buffer = Length: 67 (0x43) bytes\n0000: db 5d 01 00 00 01 00 00 00 00 00 01 04 5f 6d 67 .]..........._mg\n0010: 73 04 5f 74 63 70 0d 63 6f 6e 74 72 6f 6c 2d 70 s._tcp.control-p\n0020: 6c 61 6e 65 05 6f 78 69 64 65 08 69 6e 74 65 72 lane.oxide.inter\n0030: 6e 61 6c 00 00 21 00 01 00 00 29 04 d0 00 00 00 nal..!....).....\n0040: 00 00 00 ... kind = dns peer_addr = [::1]:61511 req_id = f4594437-3650-4435-a04d-02391fb2ce51
149462025-04-30T06:44:24.769ZDEBGtest_utilization (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 56157,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_mgs._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_mgs._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 4,\n 95,\n 109,\n 103,\n 115,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:61511 req_id = f4594437-3650-4435-a04d-02391fb2ce51
149472025-04-30T06:44:24.769ZDEBGtest_utilization (store): query key key = _mgs._tcp
149482025-04-30T06:44:24.771ZDEBGtest_utilization (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
149492025-04-30T06:44:24.771ZDEBGtest_utilization (dns): dns response additional_records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal"), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] kind = dns peer_addr = [::1]:61511 query = LowerQuery { name: LowerName(Name("_mgs._tcp.control-plane.oxide.internal.")), original: Query { name: Name("_mgs._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN } } records = [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 48759, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal") })) }] req_id = f4594437-3650-4435-a04d-02391fb2ce51
149502025-04-30T06:44:24.774ZDEBGtest_utilization (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgs._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 48759, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }], valid_until: Instant { tv_sec: 4329, tv_nsec: 573497155 } })
149512025-04-30T06:44:24.774ZTRACtest_utilization (DnsResolver): lookup_service_targets: looking up SRV target
149522025-04-30T06:44:24.775ZTRACtest_utilization (dns): buffer buffer = Length: 108 (0x6c) bytes\n0000: 13 2b 01 00 00 01 00 00 00 00 00 01 2d 64 65 6e .+..........-den\n0010: 64 72 69 74 65 2d 62 36 64 36 35 33 34 31 2d 31 drite-b6d65341-1\n0020: 36 37 63 2d 34 31 64 66 2d 39 62 35 63 2d 34 31 67c-41df-9b5c-41\n0030: 63 64 65 64 39 39 63 32 32 39 04 68 6f 73 74 0d cded99c229.host.\n0040: 63 6f 6e 74 72 6f 6c 2d 70 6c 61 6e 65 05 6f 78 control-plane.ox\n0050: 69 64 65 08 69 6e 74 65 72 6e 61 6c 00 00 1c 00 ide.internal....\n0060: 01 00 00 29 04 d0 00 00 00 00 00 00 ...)........ kind = dns peer_addr = [::1]:58894 req_id = f8d6d6ce-775a-4b30-b8b1-3ac4ab606dfe
149532025-04-30T06:44:24.775ZDEBGtest_utilization (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 4907,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n query_type: AAAA,\n query_class: IN,\n },\n },\n original: [\n 45,\n 100,\n 101,\n 110,\n 100,\n 114,\n 105,\n 116,\n 101,\n 45,\n 98,\n 54,\n 100,\n 54,\n 53,\n 51,\n 52,\n 49,\n 45,\n 49,\n 54,\n 55,\n 99,\n 45,\n 52,\n 49,\n 100,\n 102,\n 45,\n 57,\n 98,\n 53,\n 99,\n 45,\n 52,\n 49,\n 99,\n 100,\n 101,\n 100,\n 57,\n 57,\n 99,\n 50,\n 50,\n 57,\n 4,\n 104,\n 111,\n 115,\n 116,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 28,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:58894 req_id = f8d6d6ce-775a-4b30-b8b1-3ac4ab606dfe
149542025-04-30T06:44:24.775ZDEBGtest_utilization (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
149552025-04-30T06:44:24.775ZDEBGtest_utilization (dns): dns response additional_records = [] kind = dns peer_addr = [::1]:58894 query = LowerQuery { name: LowerName(Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.")), original: Query { name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), query_type: AAAA, query_class: IN } } records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] req_id = f8d6d6ce-775a-4b30-b8b1-3ac4ab606dfe
149562025-04-30T06:44:24.864ZTRACtest_utilization (DnsResolver): lookup_all_socket_v6 srv dns_name = _clickhouse-admin-keeper._tcp.control-plane.oxide.internal
149572025-04-30T06:44:24.866ZTRACtest_utilization (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory
149582025-04-30T06:44:24.866ZTRACtest_utilization (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
149592025-04-30T06:44:24.866ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
149602025-04-30T06:44:24.866ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
149612025-04-30T06:44:24.866ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
149622025-04-30T06:44:24.871ZTRACtest_utilization (dns): buffer buffer = Length: 87 (0x57) bytes\n0000: a0 6f 01 00 00 01 00 00 00 00 00 01 18 5f 63 6c .o..........._cl\n0010: 69 63 6b 68 6f 75 73 65 2d 61 64 6d 69 6e 2d 6b ickhouse-admin-k\n0020: 65 65 70 65 72 04 5f 74 63 70 0d 63 6f 6e 74 72 eeper._tcp.contr\n0030: 6f 6c 2d 70 6c 61 6e 65 05 6f 78 69 64 65 08 69 ol-plane.oxide.i\n0040: 6e 74 65 72 6e 61 6c 00 00 21 00 01 00 00 29 04 nternal..!....).\n0050: d0 00 00 00 00 00 00 ....... kind = dns peer_addr = [::1]:60847 req_id = 3d811a86-55c0-41ef-9bbf-70e77cf425e6
149632025-04-30T06:44:24.871ZDEBGtest_utilization (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 41071,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 24,\n 95,\n 99,\n 108,\n 105,\n 99,\n 107,\n 104,\n 111,\n 117,\n 115,\n 101,\n 45,\n 97,\n 100,\n 109,\n 105,\n 110,\n 45,\n 107,\n 101,\n 101,\n 112,\n 101,\n 114,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:60847 req_id = 3d811a86-55c0-41ef-9bbf-70e77cf425e6
149642025-04-30T06:44:24.871ZDEBGtest_utilization (store): query key key = _clickhouse-admin-keeper._tcp
149652025-04-30T06:44:24.871ZERROtest_utilization (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 41071, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 1, }, query: WireQuery { query: LowerQuery { name: LowerName( Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), ), original: Query { name: Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN, }, }, original: [ 24, 95, 99, 108, 105, 99, 107, 104, 111, 117, 115, 101, 45, 97, 100, 109, 105, 110, 45, 107, 101, 101, 112, 101, 114, 4, 95, 116, 99, 112, 13, 99, 111, 110, 116, 114, 111, 108, 45, 112, 108, 97, 110, 101, 5, 111, 120, 105, 100, 101, 8, 105, 110, 116, 101, 114, 110, 97, 108, 0, 0, 33, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: Some( Edns { rcode_high: 0, version: 0, dnssec_ok: false, max_payload: 1232, options: OPT { options: {}, }, }, ), } NXDOMAIN: no records found for name: "_clickhouse-admin-keeper._tcp.control-plane.oxide.internal." kind = dns peer_addr = [::1]:60847 req_id = 3d811a86-55c0-41ef-9bbf-70e77cf425e6
149662025-04-30T06:44:24.872ZDEBGtest_utilization (BackgroundTasks): begin collection background_task = inventory_collection
149672025-04-30T06:44:24.872ZDEBGtest_utilization (BackgroundTasks): begin collection from MGS background_task = inventory_collection mgs_url = http://[::1]:48759
149682025-04-30T06:44:24.873ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/ignition
149692025-04-30T06:44:24.876ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
149702025-04-30T06:44:24.884ZDEBGtest_utilization (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory result = Ok(())
149712025-04-30T06:44:24.884ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
149722025-04-30T06:44:24.884ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
149732025-04-30T06:44:24.885ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
149742025-04-30T06:44:24.886ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/ignition", status: 200, headers: {"content-type": "application/json", "x-request-id": "a8170678-d189-4f17-89ea-89997b2fff4d", "content-length": "835", "date": "Wed, 30 Apr 2025 06:44:24 GMT"} })
149752025-04-30T06:44:24.903ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0
149762025-04-30T06:44:24.908ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "0bdc8ec8-95dc-4490-9e1f-6629052702b2", "content-length": "734", "date": "Wed, 30 Apr 2025 06:44:24 GMT"} })
149772025-04-30T06:44:24.909ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0/component/sp/caboose?firmware_slot=0
149782025-04-30T06:44:24.909ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
149792025-04-30T06:44:24.909ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
149802025-04-30T06:44:24.910ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
149812025-04-30T06:44:24.912ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "0002b92e-e2ea-4061-bc92-1856412ca336", "content-length": "111", "date": "Wed, 30 Apr 2025 06:44:24 GMT"} })
149822025-04-30T06:44:24.912ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0/component/sp/caboose?firmware_slot=1
149832025-04-30T06:44:24.929ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "e306fa6b-93aa-434d-878a-9b9cc80481cc", "content-length": "111", "date": "Wed, 30 Apr 2025 06:44:24 GMT"} })
149842025-04-30T06:44:24.930ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0/component/rot/caboose?firmware_slot=0
149852025-04-30T06:44:24.935ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "81cb3408-3342-4150-b904-17bee0425c41", "content-length": "170", "date": "Wed, 30 Apr 2025 06:44:24 GMT"} })
149862025-04-30T06:44:24.935ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0/component/rot/caboose?firmware_slot=1
149872025-04-30T06:44:24.941ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "3f35f10f-7fb9-43a5-a56c-74a894dbe67a", "content-length": "170", "date": "Wed, 30 Apr 2025 06:44:24 GMT"} })
149882025-04-30T06:44:24.941ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0/component/stage0/caboose?firmware_slot=0
149892025-04-30T06:44:24.943ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "7489dff2-15fe-43ab-a491-bafd27786e2a", "content-length": "179", "date": "Wed, 30 Apr 2025 06:44:24 GMT"} })
149902025-04-30T06:44:24.943ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0/component/stage0/caboose?firmware_slot=1
149912025-04-30T06:44:24.955ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "0c7b7516-4e72-43fa-bb2b-3431373573d7", "content-length": "179", "date": "Wed, 30 Apr 2025 06:44:24 GMT"} })
149922025-04-30T06:44:24.955ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0/component/rot/cmpa
149932025-04-30T06:44:24.956ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "72b32e8e-ae64-4808-8b47-271954b26ae3", "content-length": "702", "date": "Wed, 30 Apr 2025 06:44:24 GMT"} })
149942025-04-30T06:44:24.957ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0/component/rot/cfpa
149952025-04-30T06:44:24.957ZDEBGtest_utilization: attempt to fetch latest inventory collection result = Ok("found")
149962025-04-30T06:44:24.960ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "a8e0776b-43f2-46f1-937d-5fcb9ef4a574", "content-length": "727", "date": "Wed, 30 Apr 2025 06:44:24 GMT"} })
149972025-04-30T06:44:24.960ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0/component/rot/cfpa
149982025-04-30T06:44:24.963ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "3b2b5662-bb6b-4cb2-9c97-47d813d79f65", "content-length": "729", "date": "Wed, 30 Apr 2025 06:44:24 GMT"} })
149992025-04-30T06:44:24.963ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0/component/rot/cfpa
150002025-04-30T06:44:24.965ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "fdd2fdff-fae1-43da-8379-07ac3e56f80d", "content-length": "728", "date": "Wed, 30 Apr 2025 06:44:24 GMT"} })
150012025-04-30T06:44:24.966ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1
150022025-04-30T06:44:24.967ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1", status: 200, headers: {"content-type": "application/json", "x-request-id": "3bc14258-a3ba-4940-93c8-0e0e09282d40", "content-length": "734", "date": "Wed, 30 Apr 2025 06:44:24 GMT"} })
150032025-04-30T06:44:24.968ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1/component/sp/caboose?firmware_slot=0
150042025-04-30T06:44:24.974ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "8c217f89-a9ef-48af-9367-d8e307aa8599", "content-length": "111", "date": "Wed, 30 Apr 2025 06:44:24 GMT"} })
150052025-04-30T06:44:24.975ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1/component/sp/caboose?firmware_slot=1
150062025-04-30T06:44:24.986ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "45283470-42f2-451d-b008-3cd0dff36086", "content-length": "111", "date": "Wed, 30 Apr 2025 06:44:24 GMT"} })
150072025-04-30T06:44:25.001ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1/component/rot/caboose?firmware_slot=0
150082025-04-30T06:44:25.002ZDEBGtest_utilization (collection-task): sent timer-based collection request to the collection task collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
150092025-04-30T06:44:25.003ZDEBGtest_utilization (collection-task): collecting from producer collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
150102025-04-30T06:44:25.005ZTRACtest_utilization (collection-task): sent collection request to producer collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
150112025-04-30T06:44:25.008ZDEBGtest_utilization (collection-task): collected results from producer collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 n_results = 1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
150122025-04-30T06:44:25.008ZTRACtest_utilization (collection-task): forwarded results to main collection loop collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
150132025-04-30T06:44:25.008ZTRACtest_utilization (collection-task): top of inner collection loop, waiting for next request collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
150142025-04-30T06:44:25.009ZTRACtest_utilization (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory
150152025-04-30T06:44:25.009ZTRACtest_utilization (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
150162025-04-30T06:44:25.009ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
150172025-04-30T06:44:25.009ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
150182025-04-30T06:44:25.010ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
150192025-04-30T06:44:25.011ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "c09b158e-4f31-4ea0-aebc-2e0768d03775", "content-length": "170", "date": "Wed, 30 Apr 2025 06:44:24 GMT"} })
150202025-04-30T06:44:25.011ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1/component/rot/caboose?firmware_slot=1
150212025-04-30T06:44:25.012ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
150222025-04-30T06:44:25.018ZDEBGtest_utilization (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory result = Ok(())
150232025-04-30T06:44:25.023ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
150242025-04-30T06:44:25.023ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
150252025-04-30T06:44:25.025ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
150262025-04-30T06:44:25.028ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
150272025-04-30T06:44:25.028ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
150282025-04-30T06:44:25.029ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
150292025-04-30T06:44:25.029ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "a86744f4-f547-46c9-baef-632fa59ac47b", "content-length": "170", "date": "Wed, 30 Apr 2025 06:44:24 GMT"} })
150302025-04-30T06:44:25.030ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1/component/stage0/caboose?firmware_slot=0
150312025-04-30T06:44:25.034ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "74779adf-1ce9-441d-b78f-645d90579602", "content-length": "179", "date": "Wed, 30 Apr 2025 06:44:24 GMT"} })
150322025-04-30T06:44:25.034ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1/component/stage0/caboose?firmware_slot=1
150332025-04-30T06:44:25.043ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "89419c13-eccc-4367-9f80-1f3bffc1ce43", "content-length": "179", "date": "Wed, 30 Apr 2025 06:44:24 GMT"} })
150342025-04-30T06:44:25.043ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1/component/rot/cmpa
150352025-04-30T06:44:25.044ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "3793eb31-aff1-4d56-80b6-bf4b92f99813", "content-length": "702", "date": "Wed, 30 Apr 2025 06:44:24 GMT"} })
150362025-04-30T06:44:25.045ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1/component/rot/cfpa
150372025-04-30T06:44:25.046ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "be218937-300a-4190-9933-d5f8142b07fa", "content-length": "727", "date": "Wed, 30 Apr 2025 06:44:24 GMT"} })
150382025-04-30T06:44:25.046ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1/component/rot/cfpa
150392025-04-30T06:44:25.048ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "2f6e8c2e-e1d5-4f1a-8bf9-d9cb853e0681", "content-length": "729", "date": "Wed, 30 Apr 2025 06:44:24 GMT"} })
150402025-04-30T06:44:25.048ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1/component/rot/cfpa
150412025-04-30T06:44:25.049ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "a78f66a1-61eb-45dc-bd6a-3d65a40a59e9", "content-length": "728", "date": "Wed, 30 Apr 2025 06:44:24 GMT"} })
150422025-04-30T06:44:25.049ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0
150432025-04-30T06:44:25.051ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "441056af-961e-4f38-86c5-b6e41df37e8b", "content-length": "723", "date": "Wed, 30 Apr 2025 06:44:24 GMT"} })
150442025-04-30T06:44:25.052ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0/component/sp/caboose?firmware_slot=0
150452025-04-30T06:44:25.065ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "ab1073b0-dcb8-4799-81c8-c80376740313", "content-length": "109", "date": "Wed, 30 Apr 2025 06:44:24 GMT"} })
150462025-04-30T06:44:25.065ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0/component/sp/caboose?firmware_slot=1
150472025-04-30T06:44:25.077ZDEBGtest_utilization: attempt to fetch latest inventory collection result = Ok("found")
150482025-04-30T06:44:25.078ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "455000e9-bf28-4d81-abc0-76b4ff1c3be9", "content-length": "109", "date": "Wed, 30 Apr 2025 06:44:24 GMT"} })
150492025-04-30T06:44:25.078ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0/component/rot/caboose?firmware_slot=0
150502025-04-30T06:44:25.083ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "42ac484d-35ac-44bf-9a0b-4f47156ecdb4", "content-length": "169", "date": "Wed, 30 Apr 2025 06:44:24 GMT"} })
150512025-04-30T06:44:25.083ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0/component/rot/caboose?firmware_slot=1
150522025-04-30T06:44:25.089ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "70dabd34-6d69-471f-9ab9-53c2ce09bf76", "content-length": "169", "date": "Wed, 30 Apr 2025 06:44:24 GMT"} })
150532025-04-30T06:44:25.089ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0/component/stage0/caboose?firmware_slot=0
150542025-04-30T06:44:25.092ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "026ad70d-7bcf-4e59-915e-c78308d7a5a0", "content-length": "178", "date": "Wed, 30 Apr 2025 06:44:24 GMT"} })
150552025-04-30T06:44:25.092ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0/component/stage0/caboose?firmware_slot=1
150562025-04-30T06:44:25.095ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "1c4cce56-8a15-4639-be28-fecae9353d43", "content-length": "178", "date": "Wed, 30 Apr 2025 06:44:24 GMT"} })
150572025-04-30T06:44:25.096ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0/component/rot/cmpa
150582025-04-30T06:44:25.106ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "e19ef795-f49a-4aa2-9649-1f41a045084f", "content-length": "702", "date": "Wed, 30 Apr 2025 06:44:24 GMT"} })
150592025-04-30T06:44:25.108ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0/component/rot/cfpa
150602025-04-30T06:44:25.111ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "3057a5bb-e299-41b6-9f05-e10e9eb9e2e3", "content-length": "727", "date": "Wed, 30 Apr 2025 06:44:24 GMT"} })
150612025-04-30T06:44:25.112ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0/component/rot/cfpa
150622025-04-30T06:44:25.118ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "83bd1497-af63-400c-8d97-50443814f827", "content-length": "729", "date": "Wed, 30 Apr 2025 06:44:24 GMT"} })
150632025-04-30T06:44:25.118ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0/component/rot/cfpa
150642025-04-30T06:44:25.121ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "446a5a2d-910f-4070-b163-f8508e26d278", "content-length": "728", "date": "Wed, 30 Apr 2025 06:44:24 GMT"} })
150652025-04-30T06:44:25.121ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1
150662025-04-30T06:44:25.126ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1", status: 200, headers: {"content-type": "application/json", "x-request-id": "2459195a-5fd0-4d35-b7a9-dabf0cb74611", "content-length": "723", "date": "Wed, 30 Apr 2025 06:44:24 GMT"} })
150672025-04-30T06:44:25.126ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1/component/sp/caboose?firmware_slot=0
150682025-04-30T06:44:25.128ZTRACtest_utilization (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory
150692025-04-30T06:44:25.128ZTRACtest_utilization (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
150702025-04-30T06:44:25.128ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
150712025-04-30T06:44:25.128ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
150722025-04-30T06:44:25.129ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
150732025-04-30T06:44:25.132ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
150742025-04-30T06:44:25.138ZDEBGtest_utilization (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory result = Ok(())
150752025-04-30T06:44:25.138ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
150762025-04-30T06:44:25.139ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
150772025-04-30T06:44:25.139ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
150782025-04-30T06:44:25.139ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "9f49b448-d254-4045-a59d-c54f54f54eb4", "content-length": "109", "date": "Wed, 30 Apr 2025 06:44:24 GMT"} })
150792025-04-30T06:44:25.139ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1/component/sp/caboose?firmware_slot=1
150802025-04-30T06:44:25.144ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
150812025-04-30T06:44:25.144ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
150822025-04-30T06:44:25.144ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
150832025-04-30T06:44:25.148ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "6c1a6962-eeaf-4058-967a-1d854d67acd0", "content-length": "109", "date": "Wed, 30 Apr 2025 06:44:24 GMT"} })
150842025-04-30T06:44:25.158ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1/component/rot/caboose?firmware_slot=0
150852025-04-30T06:44:25.163ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "4342f798-06b0-4b8a-ba20-9ebb66b9cde9", "content-length": "169", "date": "Wed, 30 Apr 2025 06:44:24 GMT"} })
150862025-04-30T06:44:25.163ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1/component/rot/caboose?firmware_slot=1
150872025-04-30T06:44:25.167ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "286b7ff0-ea1e-4100-9cad-74b9009b4c35", "content-length": "169", "date": "Wed, 30 Apr 2025 06:44:24 GMT"} })
150882025-04-30T06:44:25.168ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1/component/stage0/caboose?firmware_slot=0
150892025-04-30T06:44:25.171ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "84cd4c9a-142b-4d28-963b-bb0c14e884b5", "content-length": "178", "date": "Wed, 30 Apr 2025 06:44:24 GMT"} })
150902025-04-30T06:44:25.171ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1/component/stage0/caboose?firmware_slot=1
150912025-04-30T06:44:25.176ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "4ec828e8-3a1c-4bfd-902e-4c82d05679a2", "content-length": "178", "date": "Wed, 30 Apr 2025 06:44:24 GMT"} })
150922025-04-30T06:44:25.176ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1/component/rot/cmpa
150932025-04-30T06:44:25.177ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "51034ccc-2e02-4172-9298-74e3bfbf9ca6", "content-length": "702", "date": "Wed, 30 Apr 2025 06:44:24 GMT"} })
150942025-04-30T06:44:25.178ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1/component/rot/cfpa
150952025-04-30T06:44:25.179ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "646f7e79-fe52-454a-87d2-65d6f3f382d5", "content-length": "727", "date": "Wed, 30 Apr 2025 06:44:24 GMT"} })
150962025-04-30T06:44:25.179ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1/component/rot/cfpa
150972025-04-30T06:44:25.182ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "8a874b7d-b755-478e-9506-5abe4030796b", "content-length": "729", "date": "Wed, 30 Apr 2025 06:44:24 GMT"} })
150982025-04-30T06:44:25.182ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1/component/rot/cfpa
150992025-04-30T06:44:25.184ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "63b10760-c461-41bf-8bda-45e44fbd5992", "content-length": "728", "date": "Wed, 30 Apr 2025 06:44:24 GMT"} })
151002025-04-30T06:44:25.184ZTRACtest_utilization (BackgroundTasks): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet
151012025-04-30T06:44:25.184ZTRACtest_utilization (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
151022025-04-30T06:44:25.184ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
151032025-04-30T06:44:25.184ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
151042025-04-30T06:44:25.185ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
151052025-04-30T06:44:25.190ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
151062025-04-30T06:44:25.194ZDEBGtest_utilization (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet result = Ok(())
151072025-04-30T06:44:25.194ZTRACtest_utilization (BackgroundTasks): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet
151082025-04-30T06:44:25.194ZTRACtest_utilization (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
151092025-04-30T06:44:25.194ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
151102025-04-30T06:44:25.194ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
151112025-04-30T06:44:25.195ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
151122025-04-30T06:44:25.199ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
151132025-04-30T06:44:25.203ZDEBGtest_utilization (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet result = Ok(())
151142025-04-30T06:44:25.203ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
151152025-04-30T06:44:25.203ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
151162025-04-30T06:44:25.204ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
151172025-04-30T06:44:25.204ZDEBGtest_utilization: attempt to fetch latest inventory collection result = Ok("found")
151182025-04-30T06:44:25.267ZDEBGtest_utilization (BackgroundTasks): begin collection from Sled Agent background_task = inventory_collection sled_agent_url = http://[::1]:39365
151192025-04-30T06:44:25.267ZDEBGtest_utilization (BackgroundTasks): client request SledAgent = http://[::1]:39365 background_task = inventory_collection body = None method = GET uri = http://[::1]:39365/inventory
151202025-04-30T06:44:25.268ZTRACtest_utilization (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory
151212025-04-30T06:44:25.268ZTRACtest_utilization (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
151222025-04-30T06:44:25.268ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
151232025-04-30T06:44:25.268ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
151242025-04-30T06:44:25.268ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
151252025-04-30T06:44:25.268ZINFOtest_utilization (dropshot): accepted connection local_addr = [::1]:39365 remote_addr = [::1]:40726 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
151262025-04-30T06:44:25.269ZTRACtest_utilization (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal
151272025-04-30T06:44:25.270ZTRACtest_utilization (dropshot): incoming request local_addr = [::1]:39365 method = GET remote_addr = [::1]:40726 req_id = 4939fb72-f725-4af0-8e4f-2adfd2b66348 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /inventory
151282025-04-30T06:44:25.270ZINFOtest_utilization (dropshot): request completed latency_us = 880 local_addr = [::1]:39365 method = GET remote_addr = [::1]:40726 req_id = 4939fb72-f725-4af0-8e4f-2adfd2b66348 response_code = 200 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /inventory
151292025-04-30T06:44:25.272ZTRACtest_utilization (dns): buffer buffer = Length: 72 (0x48) bytes\n0000: 0b 2b 01 00 00 01 00 00 00 00 00 01 09 5f 64 65 .+..........._de\n0010: 6e 64 72 69 74 65 04 5f 74 63 70 0d 63 6f 6e 74 ndrite._tcp.cont\n0020: 72 6f 6c 2d 70 6c 61 6e 65 05 6f 78 69 64 65 08 rol-plane.oxide.\n0030: 69 6e 74 65 72 6e 61 6c 00 00 21 00 01 00 00 29 internal..!....)\n0040: 04 d0 00 00 00 00 00 00 ........ kind = dns peer_addr = [::1]:54610 req_id = 12d19ad5-8799-4e38-939f-f3a9780b08be
151302025-04-30T06:44:25.272ZDEBGtest_utilization (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 2859,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_dendrite._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_dendrite._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 9,\n 95,\n 100,\n 101,\n 110,\n 100,\n 114,\n 105,\n 116,\n 101,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:54610 req_id = 12d19ad5-8799-4e38-939f-f3a9780b08be
151312025-04-30T06:44:25.274ZDEBGtest_utilization (store): query key key = _dendrite._tcp
151322025-04-30T06:44:25.274ZDEBGtest_utilization (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
151332025-04-30T06:44:25.274ZDEBGtest_utilization (dns): dns response additional_records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal"), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] kind = dns peer_addr = [::1]:54610 query = LowerQuery { name: LowerName(Name("_dendrite._tcp.control-plane.oxide.internal.")), original: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN } } records = [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 47974, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal") })) }] req_id = 12d19ad5-8799-4e38-939f-f3a9780b08be
151342025-04-30T06:44:25.274ZDEBGtest_utilization (BackgroundTasks): client response SledAgent = http://[::1]:39365 background_task = inventory_collection result = Ok(Response { url: "http://[::1]:39365/inventory", status: 200, headers: {"content-type": "application/json", "x-request-id": "4939fb72-f725-4af0-8e4f-2adfd2b66348", "content-length": "3141", "date": "Wed, 30 Apr 2025 06:44:25 GMT"} })
151352025-04-30T06:44:25.274ZDEBGtest_utilization (BackgroundTasks): begin collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
151362025-04-30T06:44:25.274ZDEBGtest_utilization (BackgroundTasks): end collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
151372025-04-30T06:44:25.274ZDEBGtest_utilization (BackgroundTasks): finished collection background_task = inventory_collection
151382025-04-30T06:44:25.276ZTRACtest_utilization (BackgroundTasks): authorize begin action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory
151392025-04-30T06:44:25.276ZTRACtest_utilization (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
151402025-04-30T06:44:25.276ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
151412025-04-30T06:44:25.276ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
151422025-04-30T06:44:25.277ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
151432025-04-30T06:44:25.277ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
151442025-04-30T06:44:25.283ZDEBGtest_utilization (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory result = Ok(())
151452025-04-30T06:44:25.283ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
151462025-04-30T06:44:25.283ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
151472025-04-30T06:44:25.283ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
151482025-04-30T06:44:25.284ZDEBGtest_utilization (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 47974, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }], valid_until: Instant { tv_sec: 4330, tv_nsec: 83028031 } })
151492025-04-30T06:44:25.284ZTRACtest_utilization (DnsResolver): lookup_service_targets: looking up SRV target
151502025-04-30T06:44:25.292ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
151512025-04-30T06:44:25.297ZDEBGtest_utilization (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory result = Ok(())
151522025-04-30T06:44:25.297ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
151532025-04-30T06:44:25.297ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
151542025-04-30T06:44:25.298ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
151552025-04-30T06:44:25.298ZTRACtest_utilization (dns): buffer buffer = Length: 108 (0x6c) bytes\n0000: 1b 06 01 00 00 01 00 00 00 00 00 01 2d 64 65 6e ............-den\n0010: 64 72 69 74 65 2d 62 36 64 36 35 33 34 31 2d 31 drite-b6d65341-1\n0020: 36 37 63 2d 34 31 64 66 2d 39 62 35 63 2d 34 31 67c-41df-9b5c-41\n0030: 63 64 65 64 39 39 63 32 32 39 04 68 6f 73 74 0d cded99c229.host.\n0040: 63 6f 6e 74 72 6f 6c 2d 70 6c 61 6e 65 05 6f 78 control-plane.ox\n0050: 69 64 65 08 69 6e 74 65 72 6e 61 6c 00 00 1c 00 ide.internal....\n0060: 01 00 00 29 04 d0 00 00 00 00 00 00 ...)........ kind = dns peer_addr = [::1]:53686 req_id = 7d7ff72c-4442-490b-b875-e760a8786082
151562025-04-30T06:44:25.298ZDEBGtest_utilization (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 6918,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n query_type: AAAA,\n query_class: IN,\n },\n },\n original: [\n 45,\n 100,\n 101,\n 110,\n 100,\n 114,\n 105,\n 116,\n 101,\n 45,\n 98,\n 54,\n 100,\n 54,\n 53,\n 51,\n 52,\n 49,\n 45,\n 49,\n 54,\n 55,\n 99,\n 45,\n 52,\n 49,\n 100,\n 102,\n 45,\n 57,\n 98,\n 53,\n 99,\n 45,\n 52,\n 49,\n 99,\n 100,\n 101,\n 100,\n 57,\n 57,\n 99,\n 50,\n 50,\n 57,\n 4,\n 104,\n 111,\n 115,\n 116,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 28,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:53686 req_id = 7d7ff72c-4442-490b-b875-e760a8786082
151572025-04-30T06:44:25.299ZDEBGtest_utilization (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
151582025-04-30T06:44:25.299ZDEBGtest_utilization (dns): dns response additional_records = [] kind = dns peer_addr = [::1]:53686 query = LowerQuery { name: LowerName(Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.")), original: Query { name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), query_type: AAAA, query_class: IN } } records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] req_id = 7d7ff72c-4442-490b-b875-e760a8786082
151592025-04-30T06:44:25.299ZINFOtest_utilization (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_v4_garbage_collector
151602025-04-30T06:44:25.372ZINFOtest_utilization (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_v4_garbage_collector zone_address = ::1
151612025-04-30T06:44:25.372ZDEBGtest_utilization (MgsClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[::1]:12225/local/switch-id
151622025-04-30T06:44:25.374ZDEBGtest_utilization (MgsClient): client response background_task = nat_v4_garbage_collector result = Err(reqwest::Error { kind: Request, url: "http://[::1]:12225/local/switch-id", source: hyper_util::client::legacy::Error(Connect, ConnectError("tcp connect error", Os { code: 146, kind: ConnectionRefused, message: "Connection refused" })) })
151632025-04-30T06:44:25.374ZWARNtest_utilization (BackgroundTasks): failed to identify switch slot for dendrite background_task = nat_v4_garbage_collector reason = Communication Error: error sending request for url (http://[::1]:12225/local/switch-id): client error (Connect): tcp connect error: Connection refused (os error 146): Connection refused (os error 146) zone_address = ::1
151642025-04-30T06:44:25.377ZWARNtest_utilization (BackgroundTasks): Failed to map switch zone addr: Communication Error: error sending request for url (http://[::1]:12225/local/switch-id), retrying background_task = nat_v4_garbage_collector
151652025-04-30T06:44:25.379ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
151662025-04-30T06:44:25.379ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
151672025-04-30T06:44:25.379ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
151682025-04-30T06:44:25.379ZTRACtest_utilization (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal
151692025-04-30T06:44:25.379ZDEBGtest_utilization (results-sink): inserting 1 samples into database collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
151702025-04-30T06:44:25.379ZDEBGtest_utilization (results-sink-cluster): inserting 1 samples into database collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
151712025-04-30T06:44:25.380ZDEBGtest_utilization (clickhouse-client): unrolling 1 total samples collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 4d1485f0-110e-4161-9a33-c4610d68be5d
151722025-04-30T06:44:25.382ZTRACtest_utilization (clickhouse-client): inserting data collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 4d1485f0-110e-4161-9a33-c4610d68be5d n_columns = 4 n_rows = 2 sql = INSERT INTO oximeter.fields_string FORMAT Native
151732025-04-30T06:44:25.409ZTRACtest_utilization (dns): buffer buffer = Length: 72 (0x48) bytes\n0000: 85 db 01 00 00 01 00 00 00 00 00 01 09 5f 64 65 ............._de\n0010: 6e 64 72 69 74 65 04 5f 74 63 70 0d 63 6f 6e 74 ndrite._tcp.cont\n0020: 72 6f 6c 2d 70 6c 61 6e 65 05 6f 78 69 64 65 08 rol-plane.oxide.\n0030: 69 6e 74 65 72 6e 61 6c 00 00 21 00 01 00 00 29 internal..!....)\n0040: 04 d0 00 00 00 00 00 00 ........ kind = dns peer_addr = [::1]:62939 req_id = 06d7ee90-c06d-4332-ad66-a1b351440ae3
151742025-04-30T06:44:25.410ZDEBGtest_utilization (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 34267,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_dendrite._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_dendrite._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 9,\n 95,\n 100,\n 101,\n 110,\n 100,\n 114,\n 105,\n 116,\n 101,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:62939 req_id = 06d7ee90-c06d-4332-ad66-a1b351440ae3
151752025-04-30T06:44:25.410ZDEBGtest_utilization (store): query key key = _dendrite._tcp
151762025-04-30T06:44:25.410ZDEBGtest_utilization (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
151772025-04-30T06:44:25.410ZDEBGtest_utilization (dns): dns response additional_records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal"), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] kind = dns peer_addr = [::1]:62939 query = LowerQuery { name: LowerName(Name("_dendrite._tcp.control-plane.oxide.internal.")), original: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN } } records = [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 47974, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal") })) }] req_id = 06d7ee90-c06d-4332-ad66-a1b351440ae3
151782025-04-30T06:44:25.411ZDEBGtest_utilization (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 47974, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }], valid_until: Instant { tv_sec: 4330, tv_nsec: 210039503 } })
151792025-04-30T06:44:25.412ZTRACtest_utilization (DnsResolver): lookup_service_targets: looking up SRV target
151802025-04-30T06:44:25.413ZTRACtest_utilization (dns): buffer buffer = Length: 108 (0x6c) bytes\n0000: 98 ac 01 00 00 01 00 00 00 00 00 01 2d 64 65 6e ............-den\n0010: 64 72 69 74 65 2d 62 36 64 36 35 33 34 31 2d 31 drite-b6d65341-1\n0020: 36 37 63 2d 34 31 64 66 2d 39 62 35 63 2d 34 31 67c-41df-9b5c-41\n0030: 63 64 65 64 39 39 63 32 32 39 04 68 6f 73 74 0d cded99c229.host.\n0040: 63 6f 6e 74 72 6f 6c 2d 70 6c 61 6e 65 05 6f 78 control-plane.ox\n0050: 69 64 65 08 69 6e 74 65 72 6e 61 6c 00 00 1c 00 ide.internal....\n0060: 01 00 00 29 04 d0 00 00 00 00 00 00 ...)........ kind = dns peer_addr = [::1]:51633 req_id = 7183d6a2-1b52-4a4f-8ec7-fc8c7340a936
151812025-04-30T06:44:25.413ZDEBGtest_utilization (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 39084,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n query_type: AAAA,\n query_class: IN,\n },\n },\n original: [\n 45,\n 100,\n 101,\n 110,\n 100,\n 114,\n 105,\n 116,\n 101,\n 45,\n 98,\n 54,\n 100,\n 54,\n 53,\n 51,\n 52,\n 49,\n 45,\n 49,\n 54,\n 55,\n 99,\n 45,\n 52,\n 49,\n 100,\n 102,\n 45,\n 57,\n 98,\n 53,\n 99,\n 45,\n 52,\n 49,\n 99,\n 100,\n 101,\n 100,\n 57,\n 57,\n 99,\n 50,\n 50,\n 57,\n 4,\n 104,\n 111,\n 115,\n 116,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 28,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:51633 req_id = 7183d6a2-1b52-4a4f-8ec7-fc8c7340a936
151822025-04-30T06:44:25.414ZDEBGtest_utilization (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
151832025-04-30T06:44:25.414ZDEBGtest_utilization (dns): dns response additional_records = [] kind = dns peer_addr = [::1]:51633 query = LowerQuery { name: LowerName(Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.")), original: Query { name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), query_type: AAAA, query_class: IN } } records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] req_id = 7183d6a2-1b52-4a4f-8ec7-fc8c7340a936
151842025-04-30T06:44:25.414ZINFOtest_utilization (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager
151852025-04-30T06:44:25.475ZINFOtest_utilization (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager zone_address = ::1
151862025-04-30T06:44:25.475ZDEBGtest_utilization (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[::1]:12225/local/switch-id
151872025-04-30T06:44:25.475ZDEBGtest_utilization (MgsClient): client response background_task = bfd_manager result = Err(reqwest::Error { kind: Request, url: "http://[::1]:12225/local/switch-id", source: hyper_util::client::legacy::Error(Connect, ConnectError("tcp connect error", Os { code: 146, kind: ConnectionRefused, message: "Connection refused" })) })
151882025-04-30T06:44:25.484ZWARNtest_utilization (BackgroundTasks): failed to identify switch slot for dendrite background_task = bfd_manager reason = Communication Error: error sending request for url (http://[::1]:12225/local/switch-id): client error (Connect): tcp connect error: Connection refused (os error 146): Connection refused (os error 146) zone_address = ::1
151892025-04-30T06:44:25.485ZWARNtest_utilization (BackgroundTasks): Failed to map switch zone addr: Communication Error: error sending request for url (http://[::1]:12225/local/switch-id), retrying background_task = bfd_manager
151902025-04-30T06:44:25.491ZDEBGtest_utilization (clickhouse-client): inserted rows into table collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 4d1485f0-110e-4161-9a33-c4610d68be5d n_rows = 2 table_name = fields_string
151912025-04-30T06:44:25.491ZTRACtest_utilization (clickhouse-client): inserting data collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 4d1485f0-110e-4161-9a33-c4610d68be5d n_columns = 4 n_rows = 1 sql = INSERT INTO oximeter.measurements_i64 FORMAT Native
151922025-04-30T06:44:25.491ZTRACtest_utilization (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal
151932025-04-30T06:44:25.493ZWARNtest_utilization (results-sink-cluster): failed to insert some results into metric DB collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 error = Failed to check out connection to database: No backends found for this service file = oximeter/collector/src/results_sink.rs:92
151942025-04-30T06:44:25.493ZTRACtest_utilization (dns): buffer buffer = Length: 72 (0x48) bytes\n0000: 62 52 01 00 00 01 00 00 00 00 00 01 09 5f 64 65 bR..........._de\n0010: 6e 64 72 69 74 65 04 5f 74 63 70 0d 63 6f 6e 74 ndrite._tcp.cont\n0020: 72 6f 6c 2d 70 6c 61 6e 65 05 6f 78 69 64 65 08 rol-plane.oxide.\n0030: 69 6e 74 65 72 6e 61 6c 00 00 21 00 01 00 00 29 internal..!....)\n0040: 04 d0 00 00 00 00 00 00 ........ kind = dns peer_addr = [::1]:49822 req_id = cfb61312-e315-4ad8-8872-7c59994a0d9f
151952025-04-30T06:44:25.494ZDEBGtest_utilization (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 25170,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_dendrite._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_dendrite._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 9,\n 95,\n 100,\n 101,\n 110,\n 100,\n 114,\n 105,\n 116,\n 101,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:49822 req_id = cfb61312-e315-4ad8-8872-7c59994a0d9f
151962025-04-30T06:44:25.497ZDEBGtest_utilization (store): query key key = _dendrite._tcp
151972025-04-30T06:44:25.497ZDEBGtest_utilization (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
151982025-04-30T06:44:25.498ZDEBGtest_utilization (dns): dns response additional_records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal"), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] kind = dns peer_addr = [::1]:49822 query = LowerQuery { name: LowerName(Name("_dendrite._tcp.control-plane.oxide.internal.")), original: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN } } records = [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 47974, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal") })) }] req_id = cfb61312-e315-4ad8-8872-7c59994a0d9f
151992025-04-30T06:44:25.498ZDEBGtest_utilization (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 47974, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }], valid_until: Instant { tv_sec: 4330, tv_nsec: 297865010 } })
152002025-04-30T06:44:25.499ZTRACtest_utilization (DnsResolver): lookup_service_targets: looking up SRV target
152012025-04-30T06:44:25.499ZDEBGtest_utilization (clickhouse-client): inserted rows into table collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 4d1485f0-110e-4161-9a33-c4610d68be5d n_rows = 1 table_name = measurements_i64
152022025-04-30T06:44:25.499ZTRACtest_utilization (results-sink): successfully inserted samples collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
152032025-04-30T06:44:25.503ZTRACtest_utilization (dns): buffer buffer = Length: 108 (0x6c) bytes\n0000: 0f 30 01 00 00 01 00 00 00 00 00 01 2d 64 65 6e .0..........-den\n0010: 64 72 69 74 65 2d 62 36 64 36 35 33 34 31 2d 31 drite-b6d65341-1\n0020: 36 37 63 2d 34 31 64 66 2d 39 62 35 63 2d 34 31 67c-41df-9b5c-41\n0030: 63 64 65 64 39 39 63 32 32 39 04 68 6f 73 74 0d cded99c229.host.\n0040: 63 6f 6e 74 72 6f 6c 2d 70 6c 61 6e 65 05 6f 78 control-plane.ox\n0050: 69 64 65 08 69 6e 74 65 72 6e 61 6c 00 00 1c 00 ide.internal....\n0060: 01 00 00 29 04 d0 00 00 00 00 00 00 ...)........ kind = dns peer_addr = [::1]:52731 req_id = 2b5eb3ca-8aaa-44b3-a546-925e1b5d12fc
152042025-04-30T06:44:25.503ZDEBGtest_utilization (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 3888,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n query_type: AAAA,\n query_class: IN,\n },\n },\n original: [\n 45,\n 100,\n 101,\n 110,\n 100,\n 114,\n 105,\n 116,\n 101,\n 45,\n 98,\n 54,\n 100,\n 54,\n 53,\n 51,\n 52,\n 49,\n 45,\n 49,\n 54,\n 55,\n 99,\n 45,\n 52,\n 49,\n 100,\n 102,\n 45,\n 57,\n 98,\n 53,\n 99,\n 45,\n 52,\n 49,\n 99,\n 100,\n 101,\n 100,\n 57,\n 57,\n 99,\n 50,\n 50,\n 57,\n 4,\n 104,\n 111,\n 115,\n 116,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 28,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:52731 req_id = 2b5eb3ca-8aaa-44b3-a546-925e1b5d12fc
152052025-04-30T06:44:25.503ZDEBGtest_utilization (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
152062025-04-30T06:44:25.503ZDEBGtest_utilization (dns): dns response additional_records = [] kind = dns peer_addr = [::1]:52731 query = LowerQuery { name: LowerName(Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.")), original: Query { name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), query_type: AAAA, query_class: IN } } records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] req_id = 2b5eb3ca-8aaa-44b3-a546-925e1b5d12fc
152072025-04-30T06:44:25.504ZINFOtest_utilization (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager rack_id = c19a698f-c6f9-4a17-ae30-20d711b8f7dc
152082025-04-30T06:44:25.563ZINFOtest_utilization (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager rack_id = c19a698f-c6f9-4a17-ae30-20d711b8f7dc zone_address = ::1
152092025-04-30T06:44:25.563ZDEBGtest_utilization (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = c19a698f-c6f9-4a17-ae30-20d711b8f7dc uri = http://[::1]:12225/local/switch-id
152102025-04-30T06:44:25.564ZDEBGtest_utilization (MgsClient): client response background_task = switch_port_config_manager rack_id = c19a698f-c6f9-4a17-ae30-20d711b8f7dc result = Err(reqwest::Error { kind: Request, url: "http://[::1]:12225/local/switch-id", source: hyper_util::client::legacy::Error(Connect, ConnectError("tcp connect error", Os { code: 146, kind: ConnectionRefused, message: "Connection refused" })) })
152112025-04-30T06:44:25.564ZWARNtest_utilization (BackgroundTasks): failed to identify switch slot for dendrite background_task = switch_port_config_manager rack_id = c19a698f-c6f9-4a17-ae30-20d711b8f7dc reason = Communication Error: error sending request for url (http://[::1]:12225/local/switch-id): client error (Connect): tcp connect error: Connection refused (os error 146): Connection refused (os error 146) zone_address = ::1
152122025-04-30T06:44:25.565ZWARNtest_utilization (BackgroundTasks): Failed to map switch zone addr: Communication Error: error sending request for url (http://[::1]:12225/local/switch-id), retrying background_task = switch_port_config_manager rack_id = c19a698f-c6f9-4a17-ae30-20d711b8f7dc
152132025-04-30T06:44:25.580ZWARNtest_utilization (datastore): Non-retryable transaction failure err = DatabaseError(UniqueViolation, "duplicate key value violates unique constraint \\"inv_caboose_pkey\\"") transaction = inventory_insert_collection
152142025-04-30T06:44:25.580ZWARNtest_utilization (BackgroundTasks): inventory collection failed background_task = inventory_collection error = failed to collect inventory: saving inventory to database: Internal Error: unexpected database error: duplicate key value violates unique constraint "inv_caboose_pkey"
152152025-04-30T06:44:25.580ZDEBGtest_utilization (BackgroundTasks): activation complete background_task = inventory_collection elapsed = 862.348463ms iteration = 42 status = {"error":"failed to collect inventory: saving inventory to database: Internal Error: unexpected database error: duplicate key value violates unique constraint \\"inv_caboose_pkey\\""}
152162025-04-30T06:44:25.580ZDEBGtest_utilization (BackgroundTasks): activating background_task = inventory_collection iteration = 43 reason = Signaled
152172025-04-30T06:44:25.580ZTRACtest_utilization (BackgroundTasks): authorize begin action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory
152182025-04-30T06:44:25.580ZTRACtest_utilization (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
152192025-04-30T06:44:25.580ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
152202025-04-30T06:44:25.580ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
152212025-04-30T06:44:25.581ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
152222025-04-30T06:44:25.588ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
152232025-04-30T06:44:25.595ZDEBGtest_utilization (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory result = Ok(())
152242025-04-30T06:44:25.595ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
152252025-04-30T06:44:25.596ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
152262025-04-30T06:44:25.597ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
152272025-04-30T06:44:25.604ZDEBGtest_utilization (BackgroundTasks): inventory_prune_one: nothing eligible for removal (too few) background_task = inventory_collection candidates = [(13d8b8d9-6c6b-47a6-9190-952544edb75b, 0)]
152282025-04-30T06:44:25.605ZTRACtest_utilization (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal
152292025-04-30T06:44:25.610ZTRACtest_utilization (dns): buffer buffer = Length: 67 (0x43) bytes\n0000: 8b 2e 01 00 00 01 00 00 00 00 00 01 04 5f 6d 67 ............._mg\n0010: 73 04 5f 74 63 70 0d 63 6f 6e 74 72 6f 6c 2d 70 s._tcp.control-p\n0020: 6c 61 6e 65 05 6f 78 69 64 65 08 69 6e 74 65 72 lane.oxide.inter\n0030: 6e 61 6c 00 00 21 00 01 00 00 29 04 d0 00 00 00 nal..!....).....\n0040: 00 00 00 ... kind = dns peer_addr = [::1]:64066 req_id = bd65f201-7cb5-4509-9eae-0f56d341f101
152302025-04-30T06:44:25.610ZDEBGtest_utilization (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 35630,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_mgs._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_mgs._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 4,\n 95,\n 109,\n 103,\n 115,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:64066 req_id = bd65f201-7cb5-4509-9eae-0f56d341f101
152312025-04-30T06:44:25.611ZDEBGtest_utilization (store): query key key = _mgs._tcp
152322025-04-30T06:44:25.611ZDEBGtest_utilization (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
152332025-04-30T06:44:25.611ZDEBGtest_utilization (dns): dns response additional_records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal"), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] kind = dns peer_addr = [::1]:64066 query = LowerQuery { name: LowerName(Name("_mgs._tcp.control-plane.oxide.internal.")), original: Query { name: Name("_mgs._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN } } records = [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 48759, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal") })) }] req_id = bd65f201-7cb5-4509-9eae-0f56d341f101
152342025-04-30T06:44:25.612ZDEBGtest_utilization (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgs._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 48759, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }], valid_until: Instant { tv_sec: 4330, tv_nsec: 411223910 } })
152352025-04-30T06:44:25.612ZTRACtest_utilization (DnsResolver): lookup_service_targets: looking up SRV target
152362025-04-30T06:44:25.627ZTRACtest_utilization (dns): buffer buffer = Length: 108 (0x6c) bytes\n0000: 3e 83 01 00 00 01 00 00 00 00 00 01 2d 64 65 6e >...........-den\n0010: 64 72 69 74 65 2d 62 36 64 36 35 33 34 31 2d 31 drite-b6d65341-1\n0020: 36 37 63 2d 34 31 64 66 2d 39 62 35 63 2d 34 31 67c-41df-9b5c-41\n0030: 63 64 65 64 39 39 63 32 32 39 04 68 6f 73 74 0d cded99c229.host.\n0040: 63 6f 6e 74 72 6f 6c 2d 70 6c 61 6e 65 05 6f 78 control-plane.ox\n0050: 69 64 65 08 69 6e 74 65 72 6e 61 6c 00 00 1c 00 ide.internal....\n0060: 01 00 00 29 04 d0 00 00 00 00 00 00 ...)........ kind = dns peer_addr = [::1]:63126 req_id = 136be227-fc5a-4de0-85e1-01316ec784ef
152372025-04-30T06:44:25.627ZDEBGtest_utilization (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 16003,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n query_type: AAAA,\n query_class: IN,\n },\n },\n original: [\n 45,\n 100,\n 101,\n 110,\n 100,\n 114,\n 105,\n 116,\n 101,\n 45,\n 98,\n 54,\n 100,\n 54,\n 53,\n 51,\n 52,\n 49,\n 45,\n 49,\n 54,\n 55,\n 99,\n 45,\n 52,\n 49,\n 100,\n 102,\n 45,\n 57,\n 98,\n 53,\n 99,\n 45,\n 52,\n 49,\n 99,\n 100,\n 101,\n 100,\n 57,\n 57,\n 99,\n 50,\n 50,\n 57,\n 4,\n 104,\n 111,\n 115,\n 116,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 28,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:63126 req_id = 136be227-fc5a-4de0-85e1-01316ec784ef
152382025-04-30T06:44:25.627ZDEBGtest_utilization (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
152392025-04-30T06:44:25.627ZDEBGtest_utilization (dns): dns response additional_records = [] kind = dns peer_addr = [::1]:63126 query = LowerQuery { name: LowerName(Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.")), original: Query { name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), query_type: AAAA, query_class: IN } } records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] req_id = 136be227-fc5a-4de0-85e1-01316ec784ef
152402025-04-30T06:44:25.628ZDEBGtest_utilization: attempt to fetch latest inventory collection result = Ok("found")
152412025-04-30T06:44:25.675ZTRACtest_utilization (DnsResolver): lookup_all_socket_v6 srv dns_name = _clickhouse-admin-keeper._tcp.control-plane.oxide.internal
152422025-04-30T06:44:25.683ZTRACtest_utilization (dns): buffer buffer = Length: 87 (0x57) bytes\n0000: d5 71 01 00 00 01 00 00 00 00 00 01 18 5f 63 6c .q..........._cl\n0010: 69 63 6b 68 6f 75 73 65 2d 61 64 6d 69 6e 2d 6b ickhouse-admin-k\n0020: 65 65 70 65 72 04 5f 74 63 70 0d 63 6f 6e 74 72 eeper._tcp.contr\n0030: 6f 6c 2d 70 6c 61 6e 65 05 6f 78 69 64 65 08 69 ol-plane.oxide.i\n0040: 6e 74 65 72 6e 61 6c 00 00 21 00 01 00 00 29 04 nternal..!....).\n0050: d0 00 00 00 00 00 00 ....... kind = dns peer_addr = [::1]:61123 req_id = 84ac4ed9-cd79-4696-9aec-ee1270925bc1
152432025-04-30T06:44:25.683ZDEBGtest_utilization (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 54641,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 24,\n 95,\n 99,\n 108,\n 105,\n 99,\n 107,\n 104,\n 111,\n 117,\n 115,\n 101,\n 45,\n 97,\n 100,\n 109,\n 105,\n 110,\n 45,\n 107,\n 101,\n 101,\n 112,\n 101,\n 114,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:61123 req_id = 84ac4ed9-cd79-4696-9aec-ee1270925bc1
152442025-04-30T06:44:25.683ZDEBGtest_utilization (store): query key key = _clickhouse-admin-keeper._tcp
152452025-04-30T06:44:25.683ZERROtest_utilization (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 54641, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 1, }, query: WireQuery { query: LowerQuery { name: LowerName( Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), ), original: Query { name: Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN, }, }, original: [ 24, 95, 99, 108, 105, 99, 107, 104, 111, 117, 115, 101, 45, 97, 100, 109, 105, 110, 45, 107, 101, 101, 112, 101, 114, 4, 95, 116, 99, 112, 13, 99, 111, 110, 116, 114, 111, 108, 45, 112, 108, 97, 110, 101, 5, 111, 120, 105, 100, 101, 8, 105, 110, 116, 101, 114, 110, 97, 108, 0, 0, 33, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: Some( Edns { rcode_high: 0, version: 0, dnssec_ok: false, max_payload: 1232, options: OPT { options: {}, }, }, ), } NXDOMAIN: no records found for name: "_clickhouse-admin-keeper._tcp.control-plane.oxide.internal." kind = dns peer_addr = [::1]:61123 req_id = 84ac4ed9-cd79-4696-9aec-ee1270925bc1
152462025-04-30T06:44:25.683ZTRACtest_utilization (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory
152472025-04-30T06:44:25.683ZTRACtest_utilization (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
152482025-04-30T06:44:25.683ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
152492025-04-30T06:44:25.683ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
152502025-04-30T06:44:25.684ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
152512025-04-30T06:44:25.684ZDEBGtest_utilization (BackgroundTasks): begin collection background_task = inventory_collection
152522025-04-30T06:44:25.684ZDEBGtest_utilization (BackgroundTasks): begin collection from MGS background_task = inventory_collection mgs_url = http://[::1]:48759
152532025-04-30T06:44:25.684ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/ignition
152542025-04-30T06:44:25.688ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
152552025-04-30T06:44:25.693ZDEBGtest_utilization (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory result = Ok(())
152562025-04-30T06:44:25.693ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
152572025-04-30T06:44:25.693ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
152582025-04-30T06:44:25.694ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
152592025-04-30T06:44:25.694ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/ignition", status: 200, headers: {"content-type": "application/json", "x-request-id": "5035a23c-fb29-4de7-9ca1-d18ca2aa428b", "content-length": "835", "date": "Wed, 30 Apr 2025 06:44:25 GMT"} })
152602025-04-30T06:44:25.695ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0
152612025-04-30T06:44:25.697ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "6d03fca1-3981-4fc3-94be-427536928d34", "content-length": "734", "date": "Wed, 30 Apr 2025 06:44:25 GMT"} })
152622025-04-30T06:44:25.697ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0/component/sp/caboose?firmware_slot=0
152632025-04-30T06:44:25.697ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
152642025-04-30T06:44:25.697ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
152652025-04-30T06:44:25.698ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
152662025-04-30T06:44:25.702ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "134514b3-7559-4334-be3b-1dd5dc9cac15", "content-length": "111", "date": "Wed, 30 Apr 2025 06:44:25 GMT"} })
152672025-04-30T06:44:25.702ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0/component/sp/caboose?firmware_slot=1
152682025-04-30T06:44:25.704ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "9deb16d3-5170-4356-8430-d7c2d5468025", "content-length": "111", "date": "Wed, 30 Apr 2025 06:44:25 GMT"} })
152692025-04-30T06:44:25.704ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0/component/rot/caboose?firmware_slot=0
152702025-04-30T06:44:25.706ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "5b429a3e-9cea-4dfb-9976-c5bef1d2f640", "content-length": "170", "date": "Wed, 30 Apr 2025 06:44:25 GMT"} })
152712025-04-30T06:44:25.707ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0/component/rot/caboose?firmware_slot=1
152722025-04-30T06:44:25.709ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "b795ba5d-bb17-46ec-b65b-a5b62a8504f3", "content-length": "170", "date": "Wed, 30 Apr 2025 06:44:25 GMT"} })
152732025-04-30T06:44:25.709ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0/component/stage0/caboose?firmware_slot=0
152742025-04-30T06:44:25.711ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "2f2ed55d-d049-4ae2-adc0-072c8609a4fb", "content-length": "179", "date": "Wed, 30 Apr 2025 06:44:25 GMT"} })
152752025-04-30T06:44:25.711ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0/component/stage0/caboose?firmware_slot=1
152762025-04-30T06:44:25.713ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "6c51849a-cf37-4cac-ab1e-3a57169581bc", "content-length": "179", "date": "Wed, 30 Apr 2025 06:44:25 GMT"} })
152772025-04-30T06:44:25.713ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0/component/rot/cmpa
152782025-04-30T06:44:25.714ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "9716c2d7-f98c-45b8-979e-e02ef3b2d3a7", "content-length": "702", "date": "Wed, 30 Apr 2025 06:44:25 GMT"} })
152792025-04-30T06:44:25.714ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0/component/rot/cfpa
152802025-04-30T06:44:25.715ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "8a098955-5b45-4364-9e46-0f76f67e4943", "content-length": "727", "date": "Wed, 30 Apr 2025 06:44:25 GMT"} })
152812025-04-30T06:44:25.715ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0/component/rot/cfpa
152822025-04-30T06:44:25.716ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "30182a54-41a3-4635-ae25-b574b9f3e16c", "content-length": "729", "date": "Wed, 30 Apr 2025 06:44:25 GMT"} })
152832025-04-30T06:44:25.716ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0/component/rot/cfpa
152842025-04-30T06:44:25.717ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "60f07651-59b1-4665-983e-3de3bd68a0a0", "content-length": "728", "date": "Wed, 30 Apr 2025 06:44:25 GMT"} })
152852025-04-30T06:44:25.717ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1
152862025-04-30T06:44:25.719ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1", status: 200, headers: {"content-type": "application/json", "x-request-id": "7e7d888d-e314-436c-ad5f-a3befa66d72d", "content-length": "734", "date": "Wed, 30 Apr 2025 06:44:25 GMT"} })
152872025-04-30T06:44:25.719ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1/component/sp/caboose?firmware_slot=0
152882025-04-30T06:44:25.721ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "59f1b531-d437-49bf-bc53-0c2caadc5bd4", "content-length": "111", "date": "Wed, 30 Apr 2025 06:44:25 GMT"} })
152892025-04-30T06:44:25.721ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1/component/sp/caboose?firmware_slot=1
152902025-04-30T06:44:25.723ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "ab4817c0-e565-4d91-8fff-1580066fabbf", "content-length": "111", "date": "Wed, 30 Apr 2025 06:44:25 GMT"} })
152912025-04-30T06:44:25.723ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1/component/rot/caboose?firmware_slot=0
152922025-04-30T06:44:25.728ZDEBGtest_utilization: attempt to fetch latest inventory collection result = Ok("found")
152932025-04-30T06:44:25.730ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "c0507f5d-36a8-4585-9204-07bbfe1e8f5e", "content-length": "170", "date": "Wed, 30 Apr 2025 06:44:25 GMT"} })
152942025-04-30T06:44:25.730ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1/component/rot/caboose?firmware_slot=1
152952025-04-30T06:44:25.733ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "ae7084d7-e97f-487a-b415-c54d3057cbad", "content-length": "170", "date": "Wed, 30 Apr 2025 06:44:25 GMT"} })
152962025-04-30T06:44:25.733ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1/component/stage0/caboose?firmware_slot=0
152972025-04-30T06:44:25.734ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "e150d8f6-7f10-443e-a6e1-6665bef543d8", "content-length": "179", "date": "Wed, 30 Apr 2025 06:44:25 GMT"} })
152982025-04-30T06:44:25.735ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1/component/stage0/caboose?firmware_slot=1
152992025-04-30T06:44:25.739ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "4bf53355-5ca4-4176-a2cb-72529a576fbf", "content-length": "179", "date": "Wed, 30 Apr 2025 06:44:25 GMT"} })
153002025-04-30T06:44:25.739ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1/component/rot/cmpa
153012025-04-30T06:44:25.739ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "4e4e770e-4e5f-41f3-aba8-503651bbfea2", "content-length": "702", "date": "Wed, 30 Apr 2025 06:44:25 GMT"} })
153022025-04-30T06:44:25.739ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1/component/rot/cfpa
153032025-04-30T06:44:25.740ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "057d34b1-e2c5-40bf-a86d-8528d61c44f2", "content-length": "727", "date": "Wed, 30 Apr 2025 06:44:25 GMT"} })
153042025-04-30T06:44:25.741ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1/component/rot/cfpa
153052025-04-30T06:44:25.742ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "3ba55036-ff6f-4500-a804-5f1d7ea00181", "content-length": "729", "date": "Wed, 30 Apr 2025 06:44:25 GMT"} })
153062025-04-30T06:44:25.742ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1/component/rot/cfpa
153072025-04-30T06:44:25.744ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "1b6eea79-d6ba-44ec-9709-9df77ef365c8", "content-length": "728", "date": "Wed, 30 Apr 2025 06:44:25 GMT"} })
153082025-04-30T06:44:25.744ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0
153092025-04-30T06:44:25.746ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "7ed646cc-1ab1-43be-9ded-387f519beaac", "content-length": "723", "date": "Wed, 30 Apr 2025 06:44:25 GMT"} })
153102025-04-30T06:44:25.746ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0/component/sp/caboose?firmware_slot=0
153112025-04-30T06:44:25.758ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "014b1340-32ac-4af5-bde4-d530d4988527", "content-length": "109", "date": "Wed, 30 Apr 2025 06:44:25 GMT"} })
153122025-04-30T06:44:25.758ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0/component/sp/caboose?firmware_slot=1
153132025-04-30T06:44:25.764ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "c0100b99-3c07-4129-85b1-1cd9b059a1fa", "content-length": "109", "date": "Wed, 30 Apr 2025 06:44:25 GMT"} })
153142025-04-30T06:44:25.765ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0/component/rot/caboose?firmware_slot=0
153152025-04-30T06:44:25.767ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "7ce4e1d2-9902-4358-b7e7-f8382420f64f", "content-length": "169", "date": "Wed, 30 Apr 2025 06:44:25 GMT"} })
153162025-04-30T06:44:25.767ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0/component/rot/caboose?firmware_slot=1
153172025-04-30T06:44:25.768ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "40211d6d-d17d-4452-ba03-d7a4417bc0db", "content-length": "169", "date": "Wed, 30 Apr 2025 06:44:25 GMT"} })
153182025-04-30T06:44:25.769ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0/component/stage0/caboose?firmware_slot=0
153192025-04-30T06:44:25.770ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "e2b4edb2-e4fd-4afd-bdba-3f833e443e3f", "content-length": "178", "date": "Wed, 30 Apr 2025 06:44:25 GMT"} })
153202025-04-30T06:44:25.770ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0/component/stage0/caboose?firmware_slot=1
153212025-04-30T06:44:25.774ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "0df8abe7-dc9e-46ea-b429-abf12f58cf98", "content-length": "178", "date": "Wed, 30 Apr 2025 06:44:25 GMT"} })
153222025-04-30T06:44:25.774ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0/component/rot/cmpa
153232025-04-30T06:44:25.777ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "8a9e8077-b10f-4dda-89c4-4563f90d4cd7", "content-length": "702", "date": "Wed, 30 Apr 2025 06:44:25 GMT"} })
153242025-04-30T06:44:25.777ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0/component/rot/cfpa
153252025-04-30T06:44:25.778ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "87d021c4-00dd-44bd-9058-675bbf52f1f9", "content-length": "727", "date": "Wed, 30 Apr 2025 06:44:25 GMT"} })
153262025-04-30T06:44:25.778ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0/component/rot/cfpa
153272025-04-30T06:44:25.779ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "c5510bda-23cf-4ad6-9a76-b572e93f8260", "content-length": "729", "date": "Wed, 30 Apr 2025 06:44:25 GMT"} })
153282025-04-30T06:44:25.779ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0/component/rot/cfpa
153292025-04-30T06:44:25.779ZTRACtest_utilization (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory
153302025-04-30T06:44:25.779ZTRACtest_utilization (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
153312025-04-30T06:44:25.780ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
153322025-04-30T06:44:25.780ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
153332025-04-30T06:44:25.780ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
153342025-04-30T06:44:25.781ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "45651740-a6fa-4984-8d93-8b6f4c66bae2", "content-length": "728", "date": "Wed, 30 Apr 2025 06:44:25 GMT"} })
153352025-04-30T06:44:25.781ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1
153362025-04-30T06:44:25.783ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1", status: 200, headers: {"content-type": "application/json", "x-request-id": "8915ea05-2dd3-4ac8-ac18-6d7ddab40b46", "content-length": "723", "date": "Wed, 30 Apr 2025 06:44:25 GMT"} })
153372025-04-30T06:44:25.783ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1/component/sp/caboose?firmware_slot=0
153382025-04-30T06:44:25.783ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
153392025-04-30T06:44:25.797ZDEBGtest_utilization (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory result = Ok(())
153402025-04-30T06:44:25.797ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
153412025-04-30T06:44:25.797ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
153422025-04-30T06:44:25.800ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
153432025-04-30T06:44:25.803ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
153442025-04-30T06:44:25.803ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
153452025-04-30T06:44:25.803ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
153462025-04-30T06:44:25.804ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "6fb4addc-babf-4a00-af04-c777a1f4031e", "content-length": "109", "date": "Wed, 30 Apr 2025 06:44:25 GMT"} })
153472025-04-30T06:44:25.804ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1/component/sp/caboose?firmware_slot=1
153482025-04-30T06:44:25.831ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "edd6a5bd-9f3a-40b4-a0b9-67da98c97642", "content-length": "109", "date": "Wed, 30 Apr 2025 06:44:25 GMT"} })
153492025-04-30T06:44:25.832ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1/component/rot/caboose?firmware_slot=0
153502025-04-30T06:44:25.836ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "c6cb10f3-e5f1-4baf-afa1-2d3369d778ec", "content-length": "169", "date": "Wed, 30 Apr 2025 06:44:25 GMT"} })
153512025-04-30T06:44:25.836ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1/component/rot/caboose?firmware_slot=1
153522025-04-30T06:44:25.842ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "ef8c4320-122a-4e53-af62-240a3951893b", "content-length": "169", "date": "Wed, 30 Apr 2025 06:44:25 GMT"} })
153532025-04-30T06:44:25.842ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1/component/stage0/caboose?firmware_slot=0
153542025-04-30T06:44:25.847ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "366a6f8f-c673-493e-98b4-f3909ee56d49", "content-length": "178", "date": "Wed, 30 Apr 2025 06:44:25 GMT"} })
153552025-04-30T06:44:25.847ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1/component/stage0/caboose?firmware_slot=1
153562025-04-30T06:44:25.862ZDEBGtest_utilization: attempt to fetch latest inventory collection result = Ok("found")
153572025-04-30T06:44:25.862ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "f486d809-2e00-4990-8a9b-d024556b2fd0", "content-length": "178", "date": "Wed, 30 Apr 2025 06:44:25 GMT"} })
153582025-04-30T06:44:25.862ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1/component/rot/cmpa
153592025-04-30T06:44:25.864ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "00c149ce-c31f-4437-b80f-77183f455179", "content-length": "702", "date": "Wed, 30 Apr 2025 06:44:25 GMT"} })
153602025-04-30T06:44:25.864ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1/component/rot/cfpa
153612025-04-30T06:44:25.865ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "dbfc6f1a-d983-4614-8843-4be4b9336d8a", "content-length": "727", "date": "Wed, 30 Apr 2025 06:44:25 GMT"} })
153622025-04-30T06:44:25.865ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1/component/rot/cfpa
153632025-04-30T06:44:25.867ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "85498d25-050d-439d-a47a-613ba87b7d87", "content-length": "729", "date": "Wed, 30 Apr 2025 06:44:25 GMT"} })
153642025-04-30T06:44:25.868ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1/component/rot/cfpa
153652025-04-30T06:44:25.869ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "eaf3e476-64ed-4845-858a-d75b9014de85", "content-length": "728", "date": "Wed, 30 Apr 2025 06:44:25 GMT"} })
153662025-04-30T06:44:25.869ZTRACtest_utilization (BackgroundTasks): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet
153672025-04-30T06:44:25.869ZTRACtest_utilization (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
153682025-04-30T06:44:25.869ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
153692025-04-30T06:44:25.869ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
153702025-04-30T06:44:25.871ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
153712025-04-30T06:44:25.875ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
153722025-04-30T06:44:25.878ZDEBGtest_utilization (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet result = Ok(())
153732025-04-30T06:44:25.878ZTRACtest_utilization (BackgroundTasks): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet
153742025-04-30T06:44:25.878ZTRACtest_utilization (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
153752025-04-30T06:44:25.878ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
153762025-04-30T06:44:25.878ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
153772025-04-30T06:44:25.878ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
153782025-04-30T06:44:25.882ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
153792025-04-30T06:44:25.886ZDEBGtest_utilization (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet result = Ok(())
153802025-04-30T06:44:25.886ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
153812025-04-30T06:44:25.886ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
153822025-04-30T06:44:25.886ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
153832025-04-30T06:44:25.999ZDEBGtest_utilization (BackgroundTasks): begin collection from Sled Agent background_task = inventory_collection sled_agent_url = http://[::1]:39365
153842025-04-30T06:44:25.999ZDEBGtest_utilization (BackgroundTasks): client request SledAgent = http://[::1]:39365 background_task = inventory_collection body = None method = GET uri = http://[::1]:39365/inventory
153852025-04-30T06:44:26.015ZTRACtest_utilization (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory
153862025-04-30T06:44:26.015ZTRACtest_utilization (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
153872025-04-30T06:44:26.015ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
153882025-04-30T06:44:26.015ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
153892025-04-30T06:44:26.016ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
153902025-04-30T06:44:26.016ZINFOtest_utilization (dropshot): accepted connection local_addr = [::1]:39365 remote_addr = [::1]:63531 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
153912025-04-30T06:44:26.016ZINFOtest_utilization (oximeter-agent): refreshing list of producers from Nexus collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 file = oximeter/collector/src/agent.rs:562
153922025-04-30T06:44:26.019ZDEBGtest_utilization (collection-task): sent timer-based collection request to the collection task collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
153932025-04-30T06:44:26.020ZDEBGtest_utilization (collection-task): collecting from producer collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
153942025-04-30T06:44:26.021ZDEBGtest_utilization (oximeter): client request body = None method = GET uri = http://127.0.0.1:32956/metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?sort_by=id_ascending
153952025-04-30T06:44:26.022ZTRACtest_utilization (dropshot_internal): incoming request local_addr = 127.0.0.1:32956 method = GET remote_addr = 127.0.0.1:55797 req_id = 6ea5fb65-8b1c-4e9f-98cf-b055675ba678 uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?sort_by=id_ascending
153962025-04-30T06:44:26.023ZTRACtest_utilization (dropshot): incoming request local_addr = [::1]:39365 method = GET remote_addr = [::1]:63531 req_id = 45d26764-de4d-47a2-86c2-305b29872e82 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /inventory
153972025-04-30T06:44:26.025ZTRACtest_utilization (dropshot_internal): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = 127.0.0.1:32956 method = GET remote_addr = 127.0.0.1:55797 req_id = 6ea5fb65-8b1c-4e9f-98cf-b055675ba678 resource = Database uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?sort_by=id_ascending
153982025-04-30T06:44:26.025ZDEBGtest_utilization (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = 127.0.0.1:32956 method = GET remote_addr = 127.0.0.1:55797 req_id = 6ea5fb65-8b1c-4e9f-98cf-b055675ba678 roles = RoleSet { roles: {} } uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?sort_by=id_ascending
153992025-04-30T06:44:26.029ZDEBGtest_utilization (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = 127.0.0.1:32956 method = GET remote_addr = 127.0.0.1:55797 req_id = 6ea5fb65-8b1c-4e9f-98cf-b055675ba678 resource = Database result = Ok(()) uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?sort_by=id_ascending
154002025-04-30T06:44:26.035ZINFOtest_utilization (dropshot): request completed latency_us = 12661 local_addr = [::1]:39365 method = GET remote_addr = [::1]:63531 req_id = 45d26764-de4d-47a2-86c2-305b29872e82 response_code = 200 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /inventory
154012025-04-30T06:44:26.036ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
154022025-04-30T06:44:26.043ZDEBGtest_utilization (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory result = Ok(())
154032025-04-30T06:44:26.043ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
154042025-04-30T06:44:26.044ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
154052025-04-30T06:44:26.044ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
154062025-04-30T06:44:26.045ZDEBGtest_utilization (BackgroundTasks): client response SledAgent = http://[::1]:39365 background_task = inventory_collection result = Ok(Response { url: "http://[::1]:39365/inventory", status: 200, headers: {"content-type": "application/json", "x-request-id": "45d26764-de4d-47a2-86c2-305b29872e82", "content-length": "3141", "date": "Wed, 30 Apr 2025 06:44:25 GMT"} })
154072025-04-30T06:44:26.051ZDEBGtest_utilization (BackgroundTasks): begin collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
154082025-04-30T06:44:26.051ZDEBGtest_utilization (BackgroundTasks): end collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
154092025-04-30T06:44:26.051ZDEBGtest_utilization (BackgroundTasks): finished collection background_task = inventory_collection
154102025-04-30T06:44:26.053ZTRACtest_utilization (BackgroundTasks): authorize begin action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory
154112025-04-30T06:44:26.053ZTRACtest_utilization (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
154122025-04-30T06:44:26.053ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
154132025-04-30T06:44:26.054ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
154142025-04-30T06:44:26.054ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
154152025-04-30T06:44:26.058ZTRACtest_utilization (collection-task): sent collection request to producer collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
154162025-04-30T06:44:26.060ZDEBGtest_utilization (collection-task): collected results from producer collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 n_results = 1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
154172025-04-30T06:44:26.060ZTRACtest_utilization (collection-task): forwarded results to main collection loop collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
154182025-04-30T06:44:26.060ZTRACtest_utilization (collection-task): top of inner collection loop, waiting for next request collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
154192025-04-30T06:44:26.069ZINFOtest_utilization (dropshot_internal): request completed latency_us = 46752 local_addr = 127.0.0.1:32956 method = GET remote_addr = 127.0.0.1:55797 req_id = 6ea5fb65-8b1c-4e9f-98cf-b055675ba678 response_code = 200 uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?sort_by=id_ascending
154202025-04-30T06:44:26.070ZDEBGtest_utilization (oximeter): client response result = Ok(Response { url: "http://127.0.0.1:32956/metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?sort_by=id_ascending", status: 200, headers: {"content-type": "application/json", "x-request-id": "6ea5fb65-8b1c-4e9f-98cf-b055675ba678", "content-length": "405", "date": "Wed, 30 Apr 2025 06:44:25 GMT"} })
154212025-04-30T06:44:26.070ZDEBGtest_utilization (oximeter): client request body = None method = GET uri = http://127.0.0.1:32956/metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJhYzU1ZDhkOS1hMjk4LTQ1MzQtODk4Yi1iZWVlMWM5ZTRlNTIifX0%3D
154222025-04-30T06:44:26.074ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
154232025-04-30T06:44:26.074ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
154242025-04-30T06:44:26.074ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
154252025-04-30T06:44:26.075ZTRACtest_utilization (dropshot_internal): incoming request local_addr = 127.0.0.1:32956 method = GET remote_addr = 127.0.0.1:55797 req_id = f8c70789-c3ae-46d7-921b-c2b2fa3ff165 uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJhYzU1ZDhkOS1hMjk4LTQ1MzQtODk4Yi1iZWVlMWM5ZTRlNTIifX0%3D
154262025-04-30T06:44:26.075ZTRACtest_utilization (dropshot_internal): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = 127.0.0.1:32956 method = GET remote_addr = 127.0.0.1:55797 req_id = f8c70789-c3ae-46d7-921b-c2b2fa3ff165 resource = Database uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJhYzU1ZDhkOS1hMjk4LTQ1MzQtODk4Yi1iZWVlMWM5ZTRlNTIifX0%3D
154272025-04-30T06:44:26.075ZDEBGtest_utilization (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = 127.0.0.1:32956 method = GET remote_addr = 127.0.0.1:55797 req_id = f8c70789-c3ae-46d7-921b-c2b2fa3ff165 roles = RoleSet { roles: {} } uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJhYzU1ZDhkOS1hMjk4LTQ1MzQtODk4Yi1iZWVlMWM5ZTRlNTIifX0%3D
154282025-04-30T06:44:26.075ZDEBGtest_utilization (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = 127.0.0.1:32956 method = GET remote_addr = 127.0.0.1:55797 req_id = f8c70789-c3ae-46d7-921b-c2b2fa3ff165 resource = Database result = Ok(()) uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJhYzU1ZDhkOS1hMjk4LTQ1MzQtODk4Yi1iZWVlMWM5ZTRlNTIifX0%3D
154292025-04-30T06:44:26.078ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
154302025-04-30T06:44:26.095ZDEBGtest_utilization (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory result = Ok(())
154312025-04-30T06:44:26.098ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
154322025-04-30T06:44:26.098ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
154332025-04-30T06:44:26.102ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
154342025-04-30T06:44:26.109ZINFOtest_utilization (dropshot_internal): request completed latency_us = 34917 local_addr = 127.0.0.1:32956 method = GET remote_addr = 127.0.0.1:55797 req_id = f8c70789-c3ae-46d7-921b-c2b2fa3ff165 response_code = 200 uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJhYzU1ZDhkOS1hMjk4LTQ1MzQtODk4Yi1iZWVlMWM5ZTRlNTIifX0%3D
154352025-04-30T06:44:26.110ZDEBGtest_utilization (oximeter): client response result = Ok(Response { url: "http://127.0.0.1:32956/metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJhYzU1ZDhkOS1hMjk4LTQ1MzQtODk4Yi1iZWVlMWM5ZTRlNTIifX0%3D", status: 200, headers: {"content-type": "application/json", "x-request-id": "f8c70789-c3ae-46d7-921b-c2b2fa3ff165", "content-length": "29", "date": "Wed, 30 Apr 2025 06:44:25 GMT"} })
154362025-04-30T06:44:26.112ZTRACtest_utilization (oximeter-agent): ignoring request to update existing metric producer, since the endpoint information is the same as the existing collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
154372025-04-30T06:44:26.112ZTRACtest_utilization (oximeter-agent): ignoring request to update existing metric producer, since the endpoint information is the same as the existing collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = ac55d8d9-a298-4534-898b-beee1c9e4e52
154382025-04-30T06:44:26.112ZINFOtest_utilization (oximeter-agent): refreshed list of producers from Nexus collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 file = oximeter/collector/src/agent.rs:637 n_current_tasks = 2 n_pruned_tasks = 0
154392025-04-30T06:44:26.203ZDEBGtest_utilization: attempt to fetch latest inventory collection result = Ok("found")
154402025-04-30T06:44:26.256ZTRACtest_utilization (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory
154412025-04-30T06:44:26.256ZTRACtest_utilization (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
154422025-04-30T06:44:26.259ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
154432025-04-30T06:44:26.259ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
154442025-04-30T06:44:26.263ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
154452025-04-30T06:44:26.265ZWARNtest_utilization (datastore): Non-retryable transaction failure err = DatabaseError(UniqueViolation, "duplicate key value violates unique constraint \\"inv_caboose_pkey\\"") transaction = inventory_insert_collection
154462025-04-30T06:44:26.266ZWARNtest_utilization (BackgroundTasks): inventory collection failed background_task = inventory_collection error = failed to collect inventory: saving inventory to database: Internal Error: unexpected database error: duplicate key value violates unique constraint "inv_caboose_pkey"
154472025-04-30T06:44:26.274ZDEBGtest_utilization (BackgroundTasks): activation complete background_task = inventory_collection elapsed = 694.322121ms iteration = 43 status = {"error":"failed to collect inventory: saving inventory to database: Internal Error: unexpected database error: duplicate key value violates unique constraint \\"inv_caboose_pkey\\""}
154482025-04-30T06:44:26.274ZDEBGtest_utilization (BackgroundTasks): activating background_task = inventory_collection iteration = 44 reason = Signaled
154492025-04-30T06:44:26.274ZTRACtest_utilization (BackgroundTasks): authorize begin action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory
154502025-04-30T06:44:26.274ZTRACtest_utilization (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
154512025-04-30T06:44:26.275ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
154522025-04-30T06:44:26.275ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
154532025-04-30T06:44:26.276ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
154542025-04-30T06:44:26.278ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
154552025-04-30T06:44:26.290ZDEBGtest_utilization (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory result = Ok(())
154562025-04-30T06:44:26.291ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
154572025-04-30T06:44:26.291ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
154582025-04-30T06:44:26.291ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
154592025-04-30T06:44:26.297ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
154602025-04-30T06:44:26.303ZDEBGtest_utilization (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory result = Ok(())
154612025-04-30T06:44:26.303ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
154622025-04-30T06:44:26.303ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
154632025-04-30T06:44:26.304ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
154642025-04-30T06:44:26.312ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
154652025-04-30T06:44:26.312ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
154662025-04-30T06:44:26.313ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
154672025-04-30T06:44:26.328ZDEBGtest_utilization (BackgroundTasks): inventory_prune_one: nothing eligible for removal (too few) background_task = inventory_collection candidates = [(13d8b8d9-6c6b-47a6-9190-952544edb75b, 0)]
154682025-04-30T06:44:26.328ZTRACtest_utilization (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal
154692025-04-30T06:44:26.329ZTRACtest_utilization (dns): buffer buffer = Length: 67 (0x43) bytes\n0000: ad 62 01 00 00 01 00 00 00 00 00 01 04 5f 6d 67 .b..........._mg\n0010: 73 04 5f 74 63 70 0d 63 6f 6e 74 72 6f 6c 2d 70 s._tcp.control-p\n0020: 6c 61 6e 65 05 6f 78 69 64 65 08 69 6e 74 65 72 lane.oxide.inter\n0030: 6e 61 6c 00 00 21 00 01 00 00 29 04 d0 00 00 00 nal..!....).....\n0040: 00 00 00 ... kind = dns peer_addr = [::1]:50857 req_id = e2e6d33e-214d-4ffd-9495-9b12570ba25b
154702025-04-30T06:44:26.331ZDEBGtest_utilization (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 44386,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_mgs._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_mgs._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 4,\n 95,\n 109,\n 103,\n 115,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:50857 req_id = e2e6d33e-214d-4ffd-9495-9b12570ba25b
154712025-04-30T06:44:26.331ZDEBGtest_utilization (store): query key key = _mgs._tcp
154722025-04-30T06:44:26.331ZDEBGtest_utilization (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
154732025-04-30T06:44:26.331ZDEBGtest_utilization (dns): dns response additional_records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal"), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] kind = dns peer_addr = [::1]:50857 query = LowerQuery { name: LowerName(Name("_mgs._tcp.control-plane.oxide.internal.")), original: Query { name: Name("_mgs._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN } } records = [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 48759, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal") })) }] req_id = e2e6d33e-214d-4ffd-9495-9b12570ba25b
154742025-04-30T06:44:26.331ZDEBGtest_utilization (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgs._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 48759, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }], valid_until: Instant { tv_sec: 4331, tv_nsec: 130181266 } })
154752025-04-30T06:44:26.331ZTRACtest_utilization (DnsResolver): lookup_service_targets: looking up SRV target
154762025-04-30T06:44:26.331ZTRACtest_utilization (dns): buffer buffer = Length: 108 (0x6c) bytes\n0000: 20 c6 01 00 00 01 00 00 00 00 00 01 2d 64 65 6e ...........-den\n0010: 64 72 69 74 65 2d 62 36 64 36 35 33 34 31 2d 31 drite-b6d65341-1\n0020: 36 37 63 2d 34 31 64 66 2d 39 62 35 63 2d 34 31 67c-41df-9b5c-41\n0030: 63 64 65 64 39 39 63 32 32 39 04 68 6f 73 74 0d cded99c229.host.\n0040: 63 6f 6e 74 72 6f 6c 2d 70 6c 61 6e 65 05 6f 78 control-plane.ox\n0050: 69 64 65 08 69 6e 74 65 72 6e 61 6c 00 00 1c 00 ide.internal....\n0060: 01 00 00 29 04 d0 00 00 00 00 00 00 ...)........ kind = dns peer_addr = [::1]:49345 req_id = 18e8c15b-7159-4548-807d-d978fd52efa2
154772025-04-30T06:44:26.331ZDEBGtest_utilization (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 8390,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n query_type: AAAA,\n query_class: IN,\n },\n },\n original: [\n 45,\n 100,\n 101,\n 110,\n 100,\n 114,\n 105,\n 116,\n 101,\n 45,\n 98,\n 54,\n 100,\n 54,\n 53,\n 51,\n 52,\n 49,\n 45,\n 49,\n 54,\n 55,\n 99,\n 45,\n 52,\n 49,\n 100,\n 102,\n 45,\n 57,\n 98,\n 53,\n 99,\n 45,\n 52,\n 49,\n 99,\n 100,\n 101,\n 100,\n 57,\n 57,\n 99,\n 50,\n 50,\n 57,\n 4,\n 104,\n 111,\n 115,\n 116,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 28,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:49345 req_id = 18e8c15b-7159-4548-807d-d978fd52efa2
154782025-04-30T06:44:26.331ZDEBGtest_utilization (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
154792025-04-30T06:44:26.331ZDEBGtest_utilization (dns): dns response additional_records = [] kind = dns peer_addr = [::1]:49345 query = LowerQuery { name: LowerName(Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.")), original: Query { name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), query_type: AAAA, query_class: IN } } records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] req_id = 18e8c15b-7159-4548-807d-d978fd52efa2
154802025-04-30T06:44:26.417ZTRACtest_utilization (DnsResolver): lookup_all_socket_v6 srv dns_name = _clickhouse-admin-keeper._tcp.control-plane.oxide.internal
154812025-04-30T06:44:26.418ZDEBGtest_utilization (results-sink-cluster): inserting 1 samples into database collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
154822025-04-30T06:44:26.418ZDEBGtest_utilization (results-sink): inserting 1 samples into database collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
154832025-04-30T06:44:26.418ZDEBGtest_utilization (clickhouse-client): unrolling 1 total samples collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 4d1485f0-110e-4161-9a33-c4610d68be5d
154842025-04-30T06:44:26.419ZTRACtest_utilization (clickhouse-client): inserting data collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 4d1485f0-110e-4161-9a33-c4610d68be5d n_columns = 4 n_rows = 2 sql = INSERT INTO oximeter.fields_string FORMAT Native
154852025-04-30T06:44:26.419ZTRACtest_utilization (dns): buffer buffer = Length: 87 (0x57) bytes\n0000: c5 9d 01 00 00 01 00 00 00 00 00 01 18 5f 63 6c ............._cl\n0010: 69 63 6b 68 6f 75 73 65 2d 61 64 6d 69 6e 2d 6b ickhouse-admin-k\n0020: 65 65 70 65 72 04 5f 74 63 70 0d 63 6f 6e 74 72 eeper._tcp.contr\n0030: 6f 6c 2d 70 6c 61 6e 65 05 6f 78 69 64 65 08 69 ol-plane.oxide.i\n0040: 6e 74 65 72 6e 61 6c 00 00 21 00 01 00 00 29 04 nternal..!....).\n0050: d0 00 00 00 00 00 00 ....... kind = dns peer_addr = [::1]:62061 req_id = 308ae711-9220-437b-8509-f2bfb87c4e25
154862025-04-30T06:44:26.421ZDEBGtest_utilization (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 50589,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 24,\n 95,\n 99,\n 108,\n 105,\n 99,\n 107,\n 104,\n 111,\n 117,\n 115,\n 101,\n 45,\n 97,\n 100,\n 109,\n 105,\n 110,\n 45,\n 107,\n 101,\n 101,\n 112,\n 101,\n 114,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:62061 req_id = 308ae711-9220-437b-8509-f2bfb87c4e25
154872025-04-30T06:44:26.421ZDEBGtest_utilization (store): query key key = _clickhouse-admin-keeper._tcp
154882025-04-30T06:44:26.421ZERROtest_utilization (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 50589, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 1, }, query: WireQuery { query: LowerQuery { name: LowerName( Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), ), original: Query { name: Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN, }, }, original: [ 24, 95, 99, 108, 105, 99, 107, 104, 111, 117, 115, 101, 45, 97, 100, 109, 105, 110, 45, 107, 101, 101, 112, 101, 114, 4, 95, 116, 99, 112, 13, 99, 111, 110, 116, 114, 111, 108, 45, 112, 108, 97, 110, 101, 5, 111, 120, 105, 100, 101, 8, 105, 110, 116, 101, 114, 110, 97, 108, 0, 0, 33, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: Some( Edns { rcode_high: 0, version: 0, dnssec_ok: false, max_payload: 1232, options: OPT { options: {}, }, }, ), } NXDOMAIN: no records found for name: "_clickhouse-admin-keeper._tcp.control-plane.oxide.internal." kind = dns peer_addr = [::1]:62061 req_id = 308ae711-9220-437b-8509-f2bfb87c4e25
154892025-04-30T06:44:26.421ZDEBGtest_utilization (BackgroundTasks): begin collection background_task = inventory_collection
154902025-04-30T06:44:26.422ZDEBGtest_utilization (BackgroundTasks): begin collection from MGS background_task = inventory_collection mgs_url = http://[::1]:48759
154912025-04-30T06:44:26.422ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/ignition
154922025-04-30T06:44:26.425ZDEBGtest_utilization (clickhouse-client): inserted rows into table collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 4d1485f0-110e-4161-9a33-c4610d68be5d n_rows = 2 table_name = fields_string
154932025-04-30T06:44:26.425ZTRACtest_utilization (clickhouse-client): inserting data collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 4d1485f0-110e-4161-9a33-c4610d68be5d n_columns = 4 n_rows = 1 sql = INSERT INTO oximeter.measurements_i64 FORMAT Native
154942025-04-30T06:44:26.426ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/ignition", status: 200, headers: {"content-type": "application/json", "x-request-id": "112058ce-3e91-49d9-b68b-06fc68c75f69", "content-length": "835", "date": "Wed, 30 Apr 2025 06:44:26 GMT"} })
154952025-04-30T06:44:26.426ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0
154962025-04-30T06:44:26.428ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "8b9655e3-7a2d-432e-aeb6-24dfe4a9c8c6", "content-length": "734", "date": "Wed, 30 Apr 2025 06:44:26 GMT"} })
154972025-04-30T06:44:26.428ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0/component/sp/caboose?firmware_slot=0
154982025-04-30T06:44:26.429ZDEBGtest_utilization (clickhouse-client): inserted rows into table collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 4d1485f0-110e-4161-9a33-c4610d68be5d n_rows = 1 table_name = measurements_i64
154992025-04-30T06:44:26.429ZTRACtest_utilization (results-sink): successfully inserted samples collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
155002025-04-30T06:44:26.432ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "0e978794-d53c-46db-a52e-7a1eeb0c07eb", "content-length": "111", "date": "Wed, 30 Apr 2025 06:44:26 GMT"} })
155012025-04-30T06:44:26.432ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0/component/sp/caboose?firmware_slot=1
155022025-04-30T06:44:26.440ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "6c9fd878-3e4a-4c8a-9682-bf1acd463eb0", "content-length": "111", "date": "Wed, 30 Apr 2025 06:44:26 GMT"} })
155032025-04-30T06:44:26.440ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0/component/rot/caboose?firmware_slot=0
155042025-04-30T06:44:26.449ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "4ee76249-3ccd-412d-88a5-12f1c5994cea", "content-length": "170", "date": "Wed, 30 Apr 2025 06:44:26 GMT"} })
155052025-04-30T06:44:26.449ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0/component/rot/caboose?firmware_slot=1
155062025-04-30T06:44:26.451ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "fcf9c3e7-89ad-4221-953e-7932bd69976b", "content-length": "170", "date": "Wed, 30 Apr 2025 06:44:26 GMT"} })
155072025-04-30T06:44:26.451ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0/component/stage0/caboose?firmware_slot=0
155082025-04-30T06:44:26.457ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "c0fd0ba2-bc6b-4399-8fae-53cef916a34c", "content-length": "179", "date": "Wed, 30 Apr 2025 06:44:26 GMT"} })
155092025-04-30T06:44:26.457ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0/component/stage0/caboose?firmware_slot=1
155102025-04-30T06:44:26.457ZDEBGtest_utilization: attempt to fetch latest inventory collection result = Ok("found")
155112025-04-30T06:44:26.461ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "1e21f8fa-b508-4bc6-89c4-efa6d577a253", "content-length": "179", "date": "Wed, 30 Apr 2025 06:44:26 GMT"} })
155122025-04-30T06:44:26.462ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0/component/rot/cmpa
155132025-04-30T06:44:26.462ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "570cf37d-b3ea-449b-8d3e-cccff865ddd4", "content-length": "702", "date": "Wed, 30 Apr 2025 06:44:26 GMT"} })
155142025-04-30T06:44:26.463ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0/component/rot/cfpa
155152025-04-30T06:44:26.463ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "7bead90d-ed4c-4220-9efb-e84ceba78763", "content-length": "727", "date": "Wed, 30 Apr 2025 06:44:26 GMT"} })
155162025-04-30T06:44:26.464ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0/component/rot/cfpa
155172025-04-30T06:44:26.466ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "cda48233-8877-40b9-986a-a829f5c878cf", "content-length": "729", "date": "Wed, 30 Apr 2025 06:44:26 GMT"} })
155182025-04-30T06:44:26.466ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0/component/rot/cfpa
155192025-04-30T06:44:26.468ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "f5151eb0-18f6-4e4f-bbcc-2c6ac929d93b", "content-length": "728", "date": "Wed, 30 Apr 2025 06:44:26 GMT"} })
155202025-04-30T06:44:26.468ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1
155212025-04-30T06:44:26.472ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1", status: 200, headers: {"content-type": "application/json", "x-request-id": "b50cf881-592a-4d7e-b482-8811a5f11e08", "content-length": "734", "date": "Wed, 30 Apr 2025 06:44:26 GMT"} })
155222025-04-30T06:44:26.472ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1/component/sp/caboose?firmware_slot=0
155232025-04-30T06:44:26.476ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "7beabd5c-1f88-439c-9601-9200616a7965", "content-length": "111", "date": "Wed, 30 Apr 2025 06:44:26 GMT"} })
155242025-04-30T06:44:26.476ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1/component/sp/caboose?firmware_slot=1
155252025-04-30T06:44:26.481ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "9d64d437-98c2-4c23-86cb-eb91b0ff4b26", "content-length": "111", "date": "Wed, 30 Apr 2025 06:44:26 GMT"} })
155262025-04-30T06:44:26.481ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1/component/rot/caboose?firmware_slot=0
155272025-04-30T06:44:26.486ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "6a366fda-b05e-487e-9257-bff24747e5cf", "content-length": "170", "date": "Wed, 30 Apr 2025 06:44:26 GMT"} })
155282025-04-30T06:44:26.486ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1/component/rot/caboose?firmware_slot=1
155292025-04-30T06:44:26.491ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "8962ac6a-6242-4367-b4bc-e9dea71d86dc", "content-length": "170", "date": "Wed, 30 Apr 2025 06:44:26 GMT"} })
155302025-04-30T06:44:26.491ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1/component/stage0/caboose?firmware_slot=0
155312025-04-30T06:44:26.496ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "92b586a4-0a98-48e2-9974-14cf75a1e608", "content-length": "179", "date": "Wed, 30 Apr 2025 06:44:26 GMT"} })
155322025-04-30T06:44:26.498ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1/component/stage0/caboose?firmware_slot=1
155332025-04-30T06:44:26.500ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "848869df-632f-47b1-88aa-5ab4385979ce", "content-length": "179", "date": "Wed, 30 Apr 2025 06:44:26 GMT"} })
155342025-04-30T06:44:26.500ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1/component/rot/cmpa
155352025-04-30T06:44:26.501ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "992ed068-ee7d-449a-b8e1-2571c36850b9", "content-length": "702", "date": "Wed, 30 Apr 2025 06:44:26 GMT"} })
155362025-04-30T06:44:26.502ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1/component/rot/cfpa
155372025-04-30T06:44:26.505ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "9017799d-d403-4df0-b37b-53809e360623", "content-length": "727", "date": "Wed, 30 Apr 2025 06:44:26 GMT"} })
155382025-04-30T06:44:26.505ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1/component/rot/cfpa
155392025-04-30T06:44:26.508ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "5f853fc9-4693-4329-b934-9c06ab6e9be9", "content-length": "729", "date": "Wed, 30 Apr 2025 06:44:26 GMT"} })
155402025-04-30T06:44:26.508ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1/component/rot/cfpa
155412025-04-30T06:44:26.511ZTRACtest_utilization (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory
155422025-04-30T06:44:26.512ZTRACtest_utilization (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
155432025-04-30T06:44:26.512ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
155442025-04-30T06:44:26.512ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
155452025-04-30T06:44:26.513ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
155462025-04-30T06:44:26.514ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "8390b689-9c7e-4dbc-bedb-c400737ab087", "content-length": "728", "date": "Wed, 30 Apr 2025 06:44:26 GMT"} })
155472025-04-30T06:44:26.514ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0
155482025-04-30T06:44:26.516ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
155492025-04-30T06:44:26.526ZDEBGtest_utilization (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory result = Ok(())
155502025-04-30T06:44:26.526ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
155512025-04-30T06:44:26.526ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
155522025-04-30T06:44:26.527ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
155532025-04-30T06:44:26.527ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "3bcc39e4-f1c6-425b-8a0f-2e41c3398bbb", "content-length": "723", "date": "Wed, 30 Apr 2025 06:44:26 GMT"} })
155542025-04-30T06:44:26.527ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0/component/sp/caboose?firmware_slot=0
155552025-04-30T06:44:26.528ZWARNtest_utilization (results-sink-cluster): failed to insert some results into metric DB collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 error = Failed to check out connection to database: No backends found for this service file = oximeter/collector/src/results_sink.rs:92
155562025-04-30T06:44:26.531ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
155572025-04-30T06:44:26.531ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
155582025-04-30T06:44:26.531ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
155592025-04-30T06:44:26.538ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "1df5d66e-0dbd-494c-92b3-7a8e1de2458e", "content-length": "109", "date": "Wed, 30 Apr 2025 06:44:26 GMT"} })
155602025-04-30T06:44:26.539ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0/component/sp/caboose?firmware_slot=1
155612025-04-30T06:44:26.543ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "5d1c3f48-99fb-460a-8c0d-ef6f3d341013", "content-length": "109", "date": "Wed, 30 Apr 2025 06:44:26 GMT"} })
155622025-04-30T06:44:26.543ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0/component/rot/caboose?firmware_slot=0
155632025-04-30T06:44:26.548ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "7ff3abc4-4568-4f82-bf7e-acbd0df975fe", "content-length": "169", "date": "Wed, 30 Apr 2025 06:44:26 GMT"} })
155642025-04-30T06:44:26.549ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0/component/rot/caboose?firmware_slot=1
155652025-04-30T06:44:26.553ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "374afedc-0f60-4074-a629-838fc6ae1f55", "content-length": "169", "date": "Wed, 30 Apr 2025 06:44:26 GMT"} })
155662025-04-30T06:44:26.553ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0/component/stage0/caboose?firmware_slot=0
155672025-04-30T06:44:26.560ZDEBGtest_utilization: attempt to fetch latest inventory collection result = Ok("found")
155682025-04-30T06:44:26.561ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "96fb0848-1ebc-4e3e-8671-13bf89f8fa48", "content-length": "178", "date": "Wed, 30 Apr 2025 06:44:26 GMT"} })
155692025-04-30T06:44:26.561ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0/component/stage0/caboose?firmware_slot=1
155702025-04-30T06:44:26.564ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "31dddd3d-3070-4c63-9cb3-14d1a7c6979b", "content-length": "178", "date": "Wed, 30 Apr 2025 06:44:26 GMT"} })
155712025-04-30T06:44:26.564ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0/component/rot/cmpa
155722025-04-30T06:44:26.564ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "91179f7f-7f14-42aa-88a1-59a330eb4a79", "content-length": "702", "date": "Wed, 30 Apr 2025 06:44:26 GMT"} })
155732025-04-30T06:44:26.564ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0/component/rot/cfpa
155742025-04-30T06:44:26.565ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "ee6ae6bc-6db8-4c24-ae47-388be44d0137", "content-length": "727", "date": "Wed, 30 Apr 2025 06:44:26 GMT"} })
155752025-04-30T06:44:26.565ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0/component/rot/cfpa
155762025-04-30T06:44:26.570ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "f107ea75-dd70-4a3f-96aa-42cb232ea1ff", "content-length": "729", "date": "Wed, 30 Apr 2025 06:44:26 GMT"} })
155772025-04-30T06:44:26.571ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0/component/rot/cfpa
155782025-04-30T06:44:26.582ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "33497bbc-1435-4533-a746-039e387ade5f", "content-length": "728", "date": "Wed, 30 Apr 2025 06:44:26 GMT"} })
155792025-04-30T06:44:26.582ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1
155802025-04-30T06:44:26.612ZTRACtest_utilization (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory
155812025-04-30T06:44:26.613ZTRACtest_utilization (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
155822025-04-30T06:44:26.613ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
155832025-04-30T06:44:26.613ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
155842025-04-30T06:44:26.614ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
155852025-04-30T06:44:26.620ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1", status: 200, headers: {"content-type": "application/json", "x-request-id": "36a46977-1e1e-4287-ae4e-ba3c2008c929", "content-length": "723", "date": "Wed, 30 Apr 2025 06:44:26 GMT"} })
155862025-04-30T06:44:26.623ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1/component/sp/caboose?firmware_slot=0
155872025-04-30T06:44:26.623ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
155882025-04-30T06:44:26.629ZDEBGtest_utilization (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory result = Ok(())
155892025-04-30T06:44:26.629ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
155902025-04-30T06:44:26.629ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
155912025-04-30T06:44:26.630ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
155922025-04-30T06:44:26.640ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
155932025-04-30T06:44:26.640ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
155942025-04-30T06:44:26.643ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
155952025-04-30T06:44:26.652ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "af9747db-8936-4c5d-b1ff-990a1a641b79", "content-length": "109", "date": "Wed, 30 Apr 2025 06:44:26 GMT"} })
155962025-04-30T06:44:26.652ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1/component/sp/caboose?firmware_slot=1
155972025-04-30T06:44:26.655ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "dd322bf1-1d1f-4540-9aa1-5e7190894f04", "content-length": "109", "date": "Wed, 30 Apr 2025 06:44:26 GMT"} })
155982025-04-30T06:44:26.772ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1/component/rot/caboose?firmware_slot=0
155992025-04-30T06:44:26.779ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "291d5ec1-a0dc-438e-b155-1c020fa27d46", "content-length": "169", "date": "Wed, 30 Apr 2025 06:44:26 GMT"} })
156002025-04-30T06:44:26.779ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1/component/rot/caboose?firmware_slot=1
156012025-04-30T06:44:26.782ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "fe452571-12cb-498e-95d7-b74695ac6d8f", "content-length": "169", "date": "Wed, 30 Apr 2025 06:44:26 GMT"} })
156022025-04-30T06:44:26.782ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1/component/stage0/caboose?firmware_slot=0
156032025-04-30T06:44:26.786ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "a391f180-ac4d-422b-bb49-726285f0b7f0", "content-length": "178", "date": "Wed, 30 Apr 2025 06:44:26 GMT"} })
156042025-04-30T06:44:26.786ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1/component/stage0/caboose?firmware_slot=1
156052025-04-30T06:44:26.795ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "cab283e2-373d-4733-bedf-d8d3ce842bb8", "content-length": "178", "date": "Wed, 30 Apr 2025 06:44:26 GMT"} })
156062025-04-30T06:44:26.796ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1/component/rot/cmpa
156072025-04-30T06:44:26.802ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "a75569d5-cf7e-4d28-81b4-86536f478606", "content-length": "702", "date": "Wed, 30 Apr 2025 06:44:26 GMT"} })
156082025-04-30T06:44:26.804ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1/component/rot/cfpa
156092025-04-30T06:44:26.805ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "c6d69a57-7f55-45e2-ac3e-c3e506b28ec0", "content-length": "727", "date": "Wed, 30 Apr 2025 06:44:26 GMT"} })
156102025-04-30T06:44:26.805ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1/component/rot/cfpa
156112025-04-30T06:44:26.807ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "11b6c18a-215e-48a8-b54f-15f1cd77411a", "content-length": "729", "date": "Wed, 30 Apr 2025 06:44:26 GMT"} })
156122025-04-30T06:44:26.807ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1/component/rot/cfpa
156132025-04-30T06:44:26.815ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "7d6ed74f-6fd7-4b43-949b-936337b441bf", "content-length": "728", "date": "Wed, 30 Apr 2025 06:44:26 GMT"} })
156142025-04-30T06:44:26.815ZTRACtest_utilization (BackgroundTasks): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet
156152025-04-30T06:44:26.819ZTRACtest_utilization (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
156162025-04-30T06:44:26.819ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
156172025-04-30T06:44:26.819ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
156182025-04-30T06:44:26.820ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
156192025-04-30T06:44:26.821ZDEBGtest_utilization: attempt to fetch latest inventory collection result = Ok("found")
156202025-04-30T06:44:26.823ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
156212025-04-30T06:44:26.830ZDEBGtest_utilization (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet result = Ok(())
156222025-04-30T06:44:26.830ZTRACtest_utilization (BackgroundTasks): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet
156232025-04-30T06:44:26.830ZTRACtest_utilization (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
156242025-04-30T06:44:26.831ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
156252025-04-30T06:44:26.831ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
156262025-04-30T06:44:26.832ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
156272025-04-30T06:44:26.837ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
156282025-04-30T06:44:26.852ZDEBGtest_utilization (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet result = Ok(())
156292025-04-30T06:44:26.852ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
156302025-04-30T06:44:26.852ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
156312025-04-30T06:44:26.853ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
156322025-04-30T06:44:26.921ZDEBGtest_utilization (BackgroundTasks): begin collection from Sled Agent background_task = inventory_collection sled_agent_url = http://[::1]:39365
156332025-04-30T06:44:26.922ZDEBGtest_utilization (BackgroundTasks): client request SledAgent = http://[::1]:39365 background_task = inventory_collection body = None method = GET uri = http://[::1]:39365/inventory
156342025-04-30T06:44:26.923ZTRACtest_utilization (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory
156352025-04-30T06:44:26.923ZTRACtest_utilization (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
156362025-04-30T06:44:26.923ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
156372025-04-30T06:44:26.923ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
156382025-04-30T06:44:26.924ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
156392025-04-30T06:44:26.925ZINFOtest_utilization (dropshot): accepted connection local_addr = [::1]:39365 remote_addr = [::1]:32944 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
156402025-04-30T06:44:26.927ZTRACtest_utilization (dropshot): incoming request local_addr = [::1]:39365 method = GET remote_addr = [::1]:32944 req_id = a2a6d2f4-a2b5-4ac2-90e2-3d77b36bdd45 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /inventory
156412025-04-30T06:44:26.927ZINFOtest_utilization (dropshot): request completed latency_us = 594 local_addr = [::1]:39365 method = GET remote_addr = [::1]:32944 req_id = a2a6d2f4-a2b5-4ac2-90e2-3d77b36bdd45 response_code = 200 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /inventory
156422025-04-30T06:44:26.932ZDEBGtest_utilization (BackgroundTasks): client response SledAgent = http://[::1]:39365 background_task = inventory_collection result = Ok(Response { url: "http://[::1]:39365/inventory", status: 200, headers: {"content-type": "application/json", "x-request-id": "a2a6d2f4-a2b5-4ac2-90e2-3d77b36bdd45", "content-length": "3141", "date": "Wed, 30 Apr 2025 06:44:26 GMT"} })
156432025-04-30T06:44:26.935ZDEBGtest_utilization (BackgroundTasks): begin collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
156442025-04-30T06:44:26.938ZDEBGtest_utilization (BackgroundTasks): end collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
156452025-04-30T06:44:26.938ZDEBGtest_utilization (BackgroundTasks): finished collection background_task = inventory_collection
156462025-04-30T06:44:26.944ZTRACtest_utilization (BackgroundTasks): authorize begin action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory
156472025-04-30T06:44:26.944ZTRACtest_utilization (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
156482025-04-30T06:44:26.944ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
156492025-04-30T06:44:26.944ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
156502025-04-30T06:44:26.948ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
156512025-04-30T06:44:26.949ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
156522025-04-30T06:44:26.957ZDEBGtest_utilization (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory result = Ok(())
156532025-04-30T06:44:26.957ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
156542025-04-30T06:44:26.957ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
156552025-04-30T06:44:26.958ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
156562025-04-30T06:44:26.960ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
156572025-04-30T06:44:26.965ZDEBGtest_utilization (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory result = Ok(())
156582025-04-30T06:44:26.965ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
156592025-04-30T06:44:26.965ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
156602025-04-30T06:44:26.966ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
156612025-04-30T06:44:26.967ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
156622025-04-30T06:44:26.967ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
156632025-04-30T06:44:26.968ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
156642025-04-30T06:44:26.996ZDEBGtest_utilization (collection-task): sent timer-based collection request to the collection task collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
156652025-04-30T06:44:26.996ZDEBGtest_utilization (collection-task): collecting from producer collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
156662025-04-30T06:44:26.999ZTRACtest_utilization (collection-task): sent collection request to producer collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
156672025-04-30T06:44:27.001ZDEBGtest_utilization (collection-task): collected results from producer collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 n_results = 1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
156682025-04-30T06:44:27.001ZTRACtest_utilization (collection-task): forwarded results to main collection loop collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
156692025-04-30T06:44:27.002ZTRACtest_utilization (collection-task): top of inner collection loop, waiting for next request collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
156702025-04-30T06:44:27.007ZWARNtest_utilization (datastore): Non-retryable transaction failure err = DatabaseError(UniqueViolation, "duplicate key value violates unique constraint \\"inv_caboose_pkey\\"") transaction = inventory_insert_collection
156712025-04-30T06:44:27.008ZWARNtest_utilization (BackgroundTasks): inventory collection failed background_task = inventory_collection error = failed to collect inventory: saving inventory to database: Internal Error: unexpected database error: duplicate key value violates unique constraint "inv_caboose_pkey"
156722025-04-30T06:44:27.008ZDEBGtest_utilization (BackgroundTasks): activation complete background_task = inventory_collection elapsed = 732.929604ms iteration = 44 status = {"error":"failed to collect inventory: saving inventory to database: Internal Error: unexpected database error: duplicate key value violates unique constraint \\"inv_caboose_pkey\\""}
156732025-04-30T06:44:27.008ZDEBGtest_utilization (BackgroundTasks): activating background_task = inventory_collection iteration = 45 reason = Signaled
156742025-04-30T06:44:27.008ZTRACtest_utilization (BackgroundTasks): authorize begin action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory
156752025-04-30T06:44:27.008ZTRACtest_utilization (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
156762025-04-30T06:44:27.008ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
156772025-04-30T06:44:27.008ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
156782025-04-30T06:44:27.009ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
156792025-04-30T06:44:27.011ZDEBGtest_utilization: attempt to fetch latest inventory collection result = Ok("found")
156802025-04-30T06:44:27.012ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
156812025-04-30T06:44:27.019ZDEBGtest_utilization (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory result = Ok(())
156822025-04-30T06:44:27.019ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
156832025-04-30T06:44:27.019ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
156842025-04-30T06:44:27.020ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
156852025-04-30T06:44:27.034ZDEBGtest_utilization (BackgroundTasks): inventory_prune_one: nothing eligible for removal (too few) background_task = inventory_collection candidates = [(13d8b8d9-6c6b-47a6-9190-952544edb75b, 0)]
156862025-04-30T06:44:27.034ZTRACtest_utilization (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal
156872025-04-30T06:44:27.035ZTRACtest_utilization (dns): buffer buffer = Length: 67 (0x43) bytes\n0000: 32 28 01 00 00 01 00 00 00 00 00 01 04 5f 6d 67 2(..........._mg\n0010: 73 04 5f 74 63 70 0d 63 6f 6e 74 72 6f 6c 2d 70 s._tcp.control-p\n0020: 6c 61 6e 65 05 6f 78 69 64 65 08 69 6e 74 65 72 lane.oxide.inter\n0030: 6e 61 6c 00 00 21 00 01 00 00 29 04 d0 00 00 00 nal..!....).....\n0040: 00 00 00 ... kind = dns peer_addr = [::1]:56939 req_id = b688f5df-def0-4e89-a974-55d9f5c50481
156882025-04-30T06:44:27.035ZDEBGtest_utilization (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 12840,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_mgs._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_mgs._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 4,\n 95,\n 109,\n 103,\n 115,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:56939 req_id = b688f5df-def0-4e89-a974-55d9f5c50481
156892025-04-30T06:44:27.036ZDEBGtest_utilization (store): query key key = _mgs._tcp
156902025-04-30T06:44:27.036ZDEBGtest_utilization (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
156912025-04-30T06:44:27.036ZDEBGtest_utilization (dns): dns response additional_records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal"), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] kind = dns peer_addr = [::1]:56939 query = LowerQuery { name: LowerName(Name("_mgs._tcp.control-plane.oxide.internal.")), original: Query { name: Name("_mgs._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN } } records = [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 48759, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal") })) }] req_id = b688f5df-def0-4e89-a974-55d9f5c50481
156922025-04-30T06:44:27.036ZDEBGtest_utilization (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgs._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 48759, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }], valid_until: Instant { tv_sec: 4331, tv_nsec: 835804113 } })
156932025-04-30T06:44:27.036ZTRACtest_utilization (DnsResolver): lookup_service_targets: looking up SRV target
156942025-04-30T06:44:27.037ZTRACtest_utilization (dns): buffer buffer = Length: 108 (0x6c) bytes\n0000: 8c 7a 01 00 00 01 00 00 00 00 00 01 2d 64 65 6e .z..........-den\n0010: 64 72 69 74 65 2d 62 36 64 36 35 33 34 31 2d 31 drite-b6d65341-1\n0020: 36 37 63 2d 34 31 64 66 2d 39 62 35 63 2d 34 31 67c-41df-9b5c-41\n0030: 63 64 65 64 39 39 63 32 32 39 04 68 6f 73 74 0d cded99c229.host.\n0040: 63 6f 6e 74 72 6f 6c 2d 70 6c 61 6e 65 05 6f 78 control-plane.ox\n0050: 69 64 65 08 69 6e 74 65 72 6e 61 6c 00 00 1c 00 ide.internal....\n0060: 01 00 00 29 04 d0 00 00 00 00 00 00 ...)........ kind = dns peer_addr = [::1]:58795 req_id = 31e99b51-64ad-4e35-8112-e8db19fe2a78
156952025-04-30T06:44:27.038ZDEBGtest_utilization (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 35962,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n query_type: AAAA,\n query_class: IN,\n },\n },\n original: [\n 45,\n 100,\n 101,\n 110,\n 100,\n 114,\n 105,\n 116,\n 101,\n 45,\n 98,\n 54,\n 100,\n 54,\n 53,\n 51,\n 52,\n 49,\n 45,\n 49,\n 54,\n 55,\n 99,\n 45,\n 52,\n 49,\n 100,\n 102,\n 45,\n 57,\n 98,\n 53,\n 99,\n 45,\n 52,\n 49,\n 99,\n 100,\n 101,\n 100,\n 57,\n 57,\n 99,\n 50,\n 50,\n 57,\n 4,\n 104,\n 111,\n 115,\n 116,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 28,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:58795 req_id = 31e99b51-64ad-4e35-8112-e8db19fe2a78
156962025-04-30T06:44:27.038ZDEBGtest_utilization (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
156972025-04-30T06:44:27.038ZDEBGtest_utilization (dns): dns response additional_records = [] kind = dns peer_addr = [::1]:58795 query = LowerQuery { name: LowerName(Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.")), original: Query { name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), query_type: AAAA, query_class: IN } } records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] req_id = 31e99b51-64ad-4e35-8112-e8db19fe2a78
156982025-04-30T06:44:27.111ZTRACtest_utilization (DnsResolver): lookup_all_socket_v6 srv dns_name = _clickhouse-admin-keeper._tcp.control-plane.oxide.internal
156992025-04-30T06:44:27.112ZTRACtest_utilization (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory
157002025-04-30T06:44:27.112ZTRACtest_utilization (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
157012025-04-30T06:44:27.112ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
157022025-04-30T06:44:27.113ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
157032025-04-30T06:44:27.113ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
157042025-04-30T06:44:27.115ZTRACtest_utilization (dns): buffer buffer = Length: 87 (0x57) bytes\n0000: 85 5a 01 00 00 01 00 00 00 00 00 01 18 5f 63 6c .Z..........._cl\n0010: 69 63 6b 68 6f 75 73 65 2d 61 64 6d 69 6e 2d 6b ickhouse-admin-k\n0020: 65 65 70 65 72 04 5f 74 63 70 0d 63 6f 6e 74 72 eeper._tcp.contr\n0030: 6f 6c 2d 70 6c 61 6e 65 05 6f 78 69 64 65 08 69 ol-plane.oxide.i\n0040: 6e 74 65 72 6e 61 6c 00 00 21 00 01 00 00 29 04 nternal..!....).\n0050: d0 00 00 00 00 00 00 ....... kind = dns peer_addr = [::1]:64697 req_id = 0ec060f1-21b1-4dce-9a8a-2852725e2be5
157052025-04-30T06:44:27.115ZDEBGtest_utilization (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 34138,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 24,\n 95,\n 99,\n 108,\n 105,\n 99,\n 107,\n 104,\n 111,\n 117,\n 115,\n 101,\n 45,\n 97,\n 100,\n 109,\n 105,\n 110,\n 45,\n 107,\n 101,\n 101,\n 112,\n 101,\n 114,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:64697 req_id = 0ec060f1-21b1-4dce-9a8a-2852725e2be5
157062025-04-30T06:44:27.115ZDEBGtest_utilization (store): query key key = _clickhouse-admin-keeper._tcp
157072025-04-30T06:44:27.115ZERROtest_utilization (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 34138, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 1, }, query: WireQuery { query: LowerQuery { name: LowerName( Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), ), original: Query { name: Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN, }, }, original: [ 24, 95, 99, 108, 105, 99, 107, 104, 111, 117, 115, 101, 45, 97, 100, 109, 105, 110, 45, 107, 101, 101, 112, 101, 114, 4, 95, 116, 99, 112, 13, 99, 111, 110, 116, 114, 111, 108, 45, 112, 108, 97, 110, 101, 5, 111, 120, 105, 100, 101, 8, 105, 110, 116, 101, 114, 110, 97, 108, 0, 0, 33, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: Some( Edns { rcode_high: 0, version: 0, dnssec_ok: false, max_payload: 1232, options: OPT { options: {}, }, }, ), } NXDOMAIN: no records found for name: "_clickhouse-admin-keeper._tcp.control-plane.oxide.internal." kind = dns peer_addr = [::1]:64697 req_id = 0ec060f1-21b1-4dce-9a8a-2852725e2be5
157082025-04-30T06:44:27.116ZDEBGtest_utilization (BackgroundTasks): begin collection background_task = inventory_collection
157092025-04-30T06:44:27.116ZDEBGtest_utilization (BackgroundTasks): begin collection from MGS background_task = inventory_collection mgs_url = http://[::1]:48759
157102025-04-30T06:44:27.116ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/ignition
157112025-04-30T06:44:27.117ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
157122025-04-30T06:44:27.123ZDEBGtest_utilization (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory result = Ok(())
157132025-04-30T06:44:27.124ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
157142025-04-30T06:44:27.124ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
157152025-04-30T06:44:27.124ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
157162025-04-30T06:44:27.127ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/ignition", status: 200, headers: {"content-type": "application/json", "x-request-id": "3357024d-bdde-416a-89e0-bcb5f711043a", "content-length": "835", "date": "Wed, 30 Apr 2025 06:44:26 GMT"} })
157172025-04-30T06:44:27.128ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0
157182025-04-30T06:44:27.128ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
157192025-04-30T06:44:27.128ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
157202025-04-30T06:44:27.129ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
157212025-04-30T06:44:27.131ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "f4d1b910-db4f-4c4b-80c3-4ed7875b2fa2", "content-length": "734", "date": "Wed, 30 Apr 2025 06:44:26 GMT"} })
157222025-04-30T06:44:27.131ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0/component/sp/caboose?firmware_slot=0
157232025-04-30T06:44:27.135ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "25f0a4bb-1a53-4c50-ac3d-c94ddeee70fa", "content-length": "111", "date": "Wed, 30 Apr 2025 06:44:26 GMT"} })
157242025-04-30T06:44:27.135ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0/component/sp/caboose?firmware_slot=1
157252025-04-30T06:44:27.142ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "f725e034-4b2a-4729-b8c7-dee76ea2a43c", "content-length": "111", "date": "Wed, 30 Apr 2025 06:44:26 GMT"} })
157262025-04-30T06:44:27.143ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0/component/rot/caboose?firmware_slot=0
157272025-04-30T06:44:27.148ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "bd1cf6f1-2fa1-496e-aaa4-87ae539451b1", "content-length": "170", "date": "Wed, 30 Apr 2025 06:44:26 GMT"} })
157282025-04-30T06:44:27.148ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0/component/rot/caboose?firmware_slot=1
157292025-04-30T06:44:27.150ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "9e16c1db-7765-4784-b90a-60bcc07940d5", "content-length": "170", "date": "Wed, 30 Apr 2025 06:44:26 GMT"} })
157302025-04-30T06:44:27.151ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0/component/stage0/caboose?firmware_slot=0
157312025-04-30T06:44:27.155ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "0adc5b36-4aea-4120-b7d1-1298ccfcdcba", "content-length": "179", "date": "Wed, 30 Apr 2025 06:44:26 GMT"} })
157322025-04-30T06:44:27.155ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0/component/stage0/caboose?firmware_slot=1
157332025-04-30T06:44:27.158ZDEBGtest_utilization: attempt to fetch latest inventory collection result = Ok("found")
157342025-04-30T06:44:27.166ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "51425af7-24fc-4413-9f37-ad7e234ce802", "content-length": "179", "date": "Wed, 30 Apr 2025 06:44:26 GMT"} })
157352025-04-30T06:44:27.166ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0/component/rot/cmpa
157362025-04-30T06:44:27.169ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "c80618a2-7f36-4221-bffe-85758787b58c", "content-length": "702", "date": "Wed, 30 Apr 2025 06:44:26 GMT"} })
157372025-04-30T06:44:27.169ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0/component/rot/cfpa
157382025-04-30T06:44:27.172ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "4caa1da3-6855-456f-875f-35deaac6db6e", "content-length": "727", "date": "Wed, 30 Apr 2025 06:44:26 GMT"} })
157392025-04-30T06:44:27.172ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0/component/rot/cfpa
157402025-04-30T06:44:27.173ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "10692fcf-14dd-4ee7-a15e-9460598f7ad8", "content-length": "729", "date": "Wed, 30 Apr 2025 06:44:26 GMT"} })
157412025-04-30T06:44:27.173ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0/component/rot/cfpa
157422025-04-30T06:44:27.174ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "cf0ba68d-cbc5-420f-a74e-a5dbb27d9639", "content-length": "728", "date": "Wed, 30 Apr 2025 06:44:26 GMT"} })
157432025-04-30T06:44:27.174ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1
157442025-04-30T06:44:27.175ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1", status: 200, headers: {"content-type": "application/json", "x-request-id": "c9355517-acd4-41c1-a925-bd5d35c00791", "content-length": "734", "date": "Wed, 30 Apr 2025 06:44:26 GMT"} })
157452025-04-30T06:44:27.176ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1/component/sp/caboose?firmware_slot=0
157462025-04-30T06:44:27.179ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "32f9358f-a249-463e-b4c9-86ccbfd4fce4", "content-length": "111", "date": "Wed, 30 Apr 2025 06:44:26 GMT"} })
157472025-04-30T06:44:27.179ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1/component/sp/caboose?firmware_slot=1
157482025-04-30T06:44:27.182ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "63dba856-1d09-4a71-80a3-4a386f424cec", "content-length": "111", "date": "Wed, 30 Apr 2025 06:44:26 GMT"} })
157492025-04-30T06:44:27.183ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1/component/rot/caboose?firmware_slot=0
157502025-04-30T06:44:27.186ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "7228e461-08bb-47a5-aea9-4e64731ace87", "content-length": "170", "date": "Wed, 30 Apr 2025 06:44:26 GMT"} })
157512025-04-30T06:44:27.187ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1/component/rot/caboose?firmware_slot=1
157522025-04-30T06:44:27.190ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "2866a5b2-cb8e-4650-9582-2244a677e75e", "content-length": "170", "date": "Wed, 30 Apr 2025 06:44:26 GMT"} })
157532025-04-30T06:44:27.190ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1/component/stage0/caboose?firmware_slot=0
157542025-04-30T06:44:27.192ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "cb953ceb-391e-43bd-bd66-190cb20d6434", "content-length": "179", "date": "Wed, 30 Apr 2025 06:44:26 GMT"} })
157552025-04-30T06:44:27.192ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1/component/stage0/caboose?firmware_slot=1
157562025-04-30T06:44:27.195ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "e8b19b98-1a99-4fba-8bd4-f6a25d9573ef", "content-length": "179", "date": "Wed, 30 Apr 2025 06:44:26 GMT"} })
157572025-04-30T06:44:27.195ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1/component/rot/cmpa
157582025-04-30T06:44:27.196ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "1a35eb0b-d01c-4079-9207-9ebe51697242", "content-length": "702", "date": "Wed, 30 Apr 2025 06:44:26 GMT"} })
157592025-04-30T06:44:27.196ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1/component/rot/cfpa
157602025-04-30T06:44:27.197ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "4d0b2d11-6a11-45b4-86b2-6fea2d48fe60", "content-length": "727", "date": "Wed, 30 Apr 2025 06:44:26 GMT"} })
157612025-04-30T06:44:27.197ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1/component/rot/cfpa
157622025-04-30T06:44:27.198ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "877d5fca-0671-46b2-8c50-824d36a8a7c1", "content-length": "729", "date": "Wed, 30 Apr 2025 06:44:26 GMT"} })
157632025-04-30T06:44:27.198ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1/component/rot/cfpa
157642025-04-30T06:44:27.200ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "ad80a980-ba3b-45e6-8ba5-b4cce09ad66f", "content-length": "728", "date": "Wed, 30 Apr 2025 06:44:26 GMT"} })
157652025-04-30T06:44:27.200ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0
157662025-04-30T06:44:27.202ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "92e2920d-1fc2-46e4-b716-9cfe6a69af53", "content-length": "723", "date": "Wed, 30 Apr 2025 06:44:26 GMT"} })
157672025-04-30T06:44:27.203ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0/component/sp/caboose?firmware_slot=0
157682025-04-30T06:44:27.205ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "ee97613b-b5ee-4757-a3de-2a1983801725", "content-length": "109", "date": "Wed, 30 Apr 2025 06:44:26 GMT"} })
157692025-04-30T06:44:27.205ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0/component/sp/caboose?firmware_slot=1
157702025-04-30T06:44:27.208ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "6c0d4969-0ca0-46ad-ba8f-1b51e85dbcc7", "content-length": "109", "date": "Wed, 30 Apr 2025 06:44:26 GMT"} })
157712025-04-30T06:44:27.208ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0/component/rot/caboose?firmware_slot=0
157722025-04-30T06:44:27.211ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "bfcc88a5-efc9-458b-8164-c36bc9285cff", "content-length": "169", "date": "Wed, 30 Apr 2025 06:44:26 GMT"} })
157732025-04-30T06:44:27.211ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0/component/rot/caboose?firmware_slot=1
157742025-04-30T06:44:27.213ZTRACtest_utilization (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory
157752025-04-30T06:44:27.213ZTRACtest_utilization (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
157762025-04-30T06:44:27.213ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
157772025-04-30T06:44:27.213ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
157782025-04-30T06:44:27.214ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
157792025-04-30T06:44:27.215ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "e04fb786-21f0-4095-ad09-89c4f4197ab9", "content-length": "169", "date": "Wed, 30 Apr 2025 06:44:26 GMT"} })
157802025-04-30T06:44:27.215ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0/component/stage0/caboose?firmware_slot=0
157812025-04-30T06:44:27.217ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
157822025-04-30T06:44:27.224ZDEBGtest_utilization (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory result = Ok(())
157832025-04-30T06:44:27.224ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
157842025-04-30T06:44:27.224ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
157852025-04-30T06:44:27.224ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
157862025-04-30T06:44:27.227ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "a25b4dd5-8673-4200-932b-dd1d85f70ed7", "content-length": "178", "date": "Wed, 30 Apr 2025 06:44:26 GMT"} })
157872025-04-30T06:44:27.228ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0/component/stage0/caboose?firmware_slot=1
157882025-04-30T06:44:27.228ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
157892025-04-30T06:44:27.228ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
157902025-04-30T06:44:27.229ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
157912025-04-30T06:44:27.232ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "7e17f184-b108-4978-8747-c9dc73f52470", "content-length": "178", "date": "Wed, 30 Apr 2025 06:44:26 GMT"} })
157922025-04-30T06:44:27.232ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0/component/rot/cmpa
157932025-04-30T06:44:27.233ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "9024aad4-ea50-4892-8668-9bf34eb9fbf0", "content-length": "702", "date": "Wed, 30 Apr 2025 06:44:26 GMT"} })
157942025-04-30T06:44:27.233ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0/component/rot/cfpa
157952025-04-30T06:44:27.237ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "ac5c9933-240b-4da3-be59-63146f705bd6", "content-length": "727", "date": "Wed, 30 Apr 2025 06:44:26 GMT"} })
157962025-04-30T06:44:27.237ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0/component/rot/cfpa
157972025-04-30T06:44:27.239ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "02aeacf0-e9ab-44fd-9531-ff6555bd6a4a", "content-length": "729", "date": "Wed, 30 Apr 2025 06:44:26 GMT"} })
157982025-04-30T06:44:27.239ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0/component/rot/cfpa
157992025-04-30T06:44:27.242ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "7e5476f1-8caf-4714-94b4-b497c0661aad", "content-length": "728", "date": "Wed, 30 Apr 2025 06:44:26 GMT"} })
158002025-04-30T06:44:27.242ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1
158012025-04-30T06:44:27.244ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1", status: 200, headers: {"content-type": "application/json", "x-request-id": "4de48997-2961-449d-ba19-32c033fdfcf0", "content-length": "723", "date": "Wed, 30 Apr 2025 06:44:26 GMT"} })
158022025-04-30T06:44:27.244ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1/component/sp/caboose?firmware_slot=0
158032025-04-30T06:44:27.248ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "448b6710-ce8e-45a4-9303-cbb55fad5699", "content-length": "109", "date": "Wed, 30 Apr 2025 06:44:26 GMT"} })
158042025-04-30T06:44:27.251ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1/component/sp/caboose?firmware_slot=1
158052025-04-30T06:44:27.261ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "5ab9b44a-2c2b-43da-81d1-6080bd0d7618", "content-length": "109", "date": "Wed, 30 Apr 2025 06:44:26 GMT"} })
158062025-04-30T06:44:27.261ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1/component/rot/caboose?firmware_slot=0
158072025-04-30T06:44:27.265ZDEBGtest_utilization: attempt to fetch latest inventory collection result = Ok("found")
158082025-04-30T06:44:27.268ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "44111dfa-3c21-4b55-ae6b-b05251cbe425", "content-length": "169", "date": "Wed, 30 Apr 2025 06:44:26 GMT"} })
158092025-04-30T06:44:27.268ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1/component/rot/caboose?firmware_slot=1
158102025-04-30T06:44:27.276ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "5c9e95f1-f8be-42b0-b967-3bf040ad20fc", "content-length": "169", "date": "Wed, 30 Apr 2025 06:44:26 GMT"} })
158112025-04-30T06:44:27.276ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1/component/stage0/caboose?firmware_slot=0
158122025-04-30T06:44:27.282ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "bb72dce1-1231-4f4d-9ea5-92b972aee887", "content-length": "178", "date": "Wed, 30 Apr 2025 06:44:26 GMT"} })
158132025-04-30T06:44:27.282ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1/component/stage0/caboose?firmware_slot=1
158142025-04-30T06:44:27.288ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "a59b2e55-703c-4e3c-a02f-9fb07e350450", "content-length": "178", "date": "Wed, 30 Apr 2025 06:44:26 GMT"} })
158152025-04-30T06:44:27.289ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1/component/rot/cmpa
158162025-04-30T06:44:27.291ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "b1607556-9c06-4ade-87c1-e41428dc6138", "content-length": "702", "date": "Wed, 30 Apr 2025 06:44:26 GMT"} })
158172025-04-30T06:44:27.291ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1/component/rot/cfpa
158182025-04-30T06:44:27.293ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "cb469d96-1f94-40c3-b7b1-dcca6892ca5d", "content-length": "727", "date": "Wed, 30 Apr 2025 06:44:26 GMT"} })
158192025-04-30T06:44:27.293ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1/component/rot/cfpa
158202025-04-30T06:44:27.295ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "5c53a624-9d49-4c3d-9d86-1c55ee360bc9", "content-length": "729", "date": "Wed, 30 Apr 2025 06:44:26 GMT"} })
158212025-04-30T06:44:27.295ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1/component/rot/cfpa
158222025-04-30T06:44:27.296ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "516dac41-a903-4988-964f-e34e48f76cfb", "content-length": "728", "date": "Wed, 30 Apr 2025 06:44:26 GMT"} })
158232025-04-30T06:44:27.296ZTRACtest_utilization (BackgroundTasks): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet
158242025-04-30T06:44:27.296ZTRACtest_utilization (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
158252025-04-30T06:44:27.296ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
158262025-04-30T06:44:27.296ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
158272025-04-30T06:44:27.297ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
158282025-04-30T06:44:27.300ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
158292025-04-30T06:44:27.308ZDEBGtest_utilization (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet result = Ok(())
158302025-04-30T06:44:27.308ZTRACtest_utilization (BackgroundTasks): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet
158312025-04-30T06:44:27.308ZTRACtest_utilization (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
158322025-04-30T06:44:27.308ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
158332025-04-30T06:44:27.308ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
158342025-04-30T06:44:27.309ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
158352025-04-30T06:44:27.312ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
158362025-04-30T06:44:27.320ZDEBGtest_utilization (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet result = Ok(())
158372025-04-30T06:44:27.320ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
158382025-04-30T06:44:27.320ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
158392025-04-30T06:44:27.321ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
158402025-04-30T06:44:27.321ZTRACtest_utilization (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory
158412025-04-30T06:44:27.321ZTRACtest_utilization (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
158422025-04-30T06:44:27.321ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
158432025-04-30T06:44:27.321ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
158442025-04-30T06:44:27.322ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
158452025-04-30T06:44:27.389ZDEBGtest_utilization (BackgroundTasks): begin collection from Sled Agent background_task = inventory_collection sled_agent_url = http://[::1]:39365
158462025-04-30T06:44:27.390ZDEBGtest_utilization (BackgroundTasks): client request SledAgent = http://[::1]:39365 background_task = inventory_collection body = None method = GET uri = http://[::1]:39365/inventory
158472025-04-30T06:44:27.390ZINFOtest_utilization (dropshot): accepted connection local_addr = [::1]:39365 remote_addr = [::1]:45227 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
158482025-04-30T06:44:27.392ZDEBGtest_utilization (results-sink): inserting 1 samples into database collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
158492025-04-30T06:44:27.392ZDEBGtest_utilization (results-sink-cluster): inserting 1 samples into database collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
158502025-04-30T06:44:27.392ZTRACtest_utilization (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal
158512025-04-30T06:44:27.393ZDEBGtest_utilization (clickhouse-client): unrolling 1 total samples collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 4d1485f0-110e-4161-9a33-c4610d68be5d
158522025-04-30T06:44:27.395ZTRACtest_utilization (clickhouse-client): inserting data collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 4d1485f0-110e-4161-9a33-c4610d68be5d n_columns = 4 n_rows = 2 sql = INSERT INTO oximeter.fields_string FORMAT Native
158532025-04-30T06:44:27.396ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
158542025-04-30T06:44:27.401ZDEBGtest_utilization (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory result = Ok(())
158552025-04-30T06:44:27.401ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
158562025-04-30T06:44:27.402ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
158572025-04-30T06:44:27.402ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
158582025-04-30T06:44:27.402ZTRACtest_utilization (dropshot): incoming request local_addr = [::1]:39365 method = GET remote_addr = [::1]:45227 req_id = be4d78de-9083-4725-a3de-eb85e73aca4e sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /inventory
158592025-04-30T06:44:27.403ZINFOtest_utilization (dropshot): request completed latency_us = 859 local_addr = [::1]:39365 method = GET remote_addr = [::1]:45227 req_id = be4d78de-9083-4725-a3de-eb85e73aca4e response_code = 200 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /inventory
158602025-04-30T06:44:27.406ZDEBGtest_utilization (clickhouse-client): inserted rows into table collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 4d1485f0-110e-4161-9a33-c4610d68be5d n_rows = 2 table_name = fields_string
158612025-04-30T06:44:27.406ZTRACtest_utilization (clickhouse-client): inserting data collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 4d1485f0-110e-4161-9a33-c4610d68be5d n_columns = 4 n_rows = 1 sql = INSERT INTO oximeter.measurements_i64 FORMAT Native
158622025-04-30T06:44:27.406ZTRACtest_utilization (dns): buffer buffer = Length: 72 (0x48) bytes\n0000: f3 66 01 00 00 01 00 00 00 00 00 01 09 5f 64 65 .f..........._de\n0010: 6e 64 72 69 74 65 04 5f 74 63 70 0d 63 6f 6e 74 ndrite._tcp.cont\n0020: 72 6f 6c 2d 70 6c 61 6e 65 05 6f 78 69 64 65 08 rol-plane.oxide.\n0030: 69 6e 74 65 72 6e 61 6c 00 00 21 00 01 00 00 29 internal..!....)\n0040: 04 d0 00 00 00 00 00 00 ........ kind = dns peer_addr = [::1]:59278 req_id = 66f2cf4f-f27f-4e8f-a54b-2e49cb667e72
158632025-04-30T06:44:27.406ZDEBGtest_utilization (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 62310,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_dendrite._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_dendrite._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 9,\n 95,\n 100,\n 101,\n 110,\n 100,\n 114,\n 105,\n 116,\n 101,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:59278 req_id = 66f2cf4f-f27f-4e8f-a54b-2e49cb667e72
158642025-04-30T06:44:27.406ZDEBGtest_utilization (store): query key key = _dendrite._tcp
158652025-04-30T06:44:27.406ZDEBGtest_utilization (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
158662025-04-30T06:44:27.407ZDEBGtest_utilization (dns): dns response additional_records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal"), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] kind = dns peer_addr = [::1]:59278 query = LowerQuery { name: LowerName(Name("_dendrite._tcp.control-plane.oxide.internal.")), original: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN } } records = [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 47974, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal") })) }] req_id = 66f2cf4f-f27f-4e8f-a54b-2e49cb667e72
158672025-04-30T06:44:27.407ZDEBGtest_utilization (BackgroundTasks): client response SledAgent = http://[::1]:39365 background_task = inventory_collection result = Ok(Response { url: "http://[::1]:39365/inventory", status: 200, headers: {"content-type": "application/json", "x-request-id": "be4d78de-9083-4725-a3de-eb85e73aca4e", "content-length": "3141", "date": "Wed, 30 Apr 2025 06:44:26 GMT"} })
158682025-04-30T06:44:27.411ZDEBGtest_utilization (BackgroundTasks): begin collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
158692025-04-30T06:44:27.411ZDEBGtest_utilization (BackgroundTasks): end collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
158702025-04-30T06:44:27.411ZDEBGtest_utilization (BackgroundTasks): finished collection background_task = inventory_collection
158712025-04-30T06:44:27.413ZTRACtest_utilization (BackgroundTasks): authorize begin action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory
158722025-04-30T06:44:27.413ZTRACtest_utilization (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
158732025-04-30T06:44:27.414ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
158742025-04-30T06:44:27.414ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
158752025-04-30T06:44:27.414ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
158762025-04-30T06:44:27.415ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
158772025-04-30T06:44:27.415ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
158782025-04-30T06:44:27.416ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
158792025-04-30T06:44:27.417ZDEBGtest_utilization (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 47974, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }], valid_until: Instant { tv_sec: 4332, tv_nsec: 216628330 } })
158802025-04-30T06:44:27.417ZTRACtest_utilization (DnsResolver): lookup_service_targets: looking up SRV target
158812025-04-30T06:44:27.418ZTRACtest_utilization (dns): buffer buffer = Length: 108 (0x6c) bytes\n0000: e8 34 01 00 00 01 00 00 00 00 00 01 2d 64 65 6e .4..........-den\n0010: 64 72 69 74 65 2d 62 36 64 36 35 33 34 31 2d 31 drite-b6d65341-1\n0020: 36 37 63 2d 34 31 64 66 2d 39 62 35 63 2d 34 31 67c-41df-9b5c-41\n0030: 63 64 65 64 39 39 63 32 32 39 04 68 6f 73 74 0d cded99c229.host.\n0040: 63 6f 6e 74 72 6f 6c 2d 70 6c 61 6e 65 05 6f 78 control-plane.ox\n0050: 69 64 65 08 69 6e 74 65 72 6e 61 6c 00 00 1c 00 ide.internal....\n0060: 01 00 00 29 04 d0 00 00 00 00 00 00 ...)........ kind = dns peer_addr = [::1]:51311 req_id = 6735dcd7-7eda-49d3-aa99-59ad0ef10352
158822025-04-30T06:44:27.418ZDEBGtest_utilization (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 59444,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n query_type: AAAA,\n query_class: IN,\n },\n },\n original: [\n 45,\n 100,\n 101,\n 110,\n 100,\n 114,\n 105,\n 116,\n 101,\n 45,\n 98,\n 54,\n 100,\n 54,\n 53,\n 51,\n 52,\n 49,\n 45,\n 49,\n 54,\n 55,\n 99,\n 45,\n 52,\n 49,\n 100,\n 102,\n 45,\n 57,\n 98,\n 53,\n 99,\n 45,\n 52,\n 49,\n 99,\n 100,\n 101,\n 100,\n 57,\n 57,\n 99,\n 50,\n 50,\n 57,\n 4,\n 104,\n 111,\n 115,\n 116,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 28,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:51311 req_id = 6735dcd7-7eda-49d3-aa99-59ad0ef10352
158832025-04-30T06:44:27.418ZDEBGtest_utilization (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
158842025-04-30T06:44:27.418ZDEBGtest_utilization (dns): dns response additional_records = [] kind = dns peer_addr = [::1]:51311 query = LowerQuery { name: LowerName(Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.")), original: Query { name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), query_type: AAAA, query_class: IN } } records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] req_id = 6735dcd7-7eda-49d3-aa99-59ad0ef10352
158852025-04-30T06:44:27.419ZINFOtest_utilization (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_v4_garbage_collector
158862025-04-30T06:44:27.464ZINFOtest_utilization (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_v4_garbage_collector zone_address = ::1
158872025-04-30T06:44:27.465ZDEBGtest_utilization (MgsClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[::1]:12225/local/switch-id
158882025-04-30T06:44:27.465ZDEBGtest_utilization (MgsClient): client response background_task = nat_v4_garbage_collector result = Err(reqwest::Error { kind: Request, url: "http://[::1]:12225/local/switch-id", source: hyper_util::client::legacy::Error(Connect, ConnectError("tcp connect error", Os { code: 146, kind: ConnectionRefused, message: "Connection refused" })) })
158892025-04-30T06:44:27.466ZWARNtest_utilization (BackgroundTasks): failed to identify switch slot for dendrite background_task = nat_v4_garbage_collector reason = Communication Error: error sending request for url (http://[::1]:12225/local/switch-id): client error (Connect): tcp connect error: Connection refused (os error 146): Connection refused (os error 146) zone_address = ::1
158902025-04-30T06:44:27.467ZWARNtest_utilization (BackgroundTasks): Failed to map switch zone addr: Communication Error: error sending request for url (http://[::1]:12225/local/switch-id), retrying background_task = nat_v4_garbage_collector
158912025-04-30T06:44:27.467ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
158922025-04-30T06:44:27.471ZDEBGtest_utilization (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory result = Ok(())
158932025-04-30T06:44:27.472ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
158942025-04-30T06:44:27.472ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
158952025-04-30T06:44:27.473ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
158962025-04-30T06:44:27.473ZDEBGtest_utilization (clickhouse-client): inserted rows into table collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 4d1485f0-110e-4161-9a33-c4610d68be5d n_rows = 1 table_name = measurements_i64
158972025-04-30T06:44:27.475ZTRACtest_utilization (results-sink): successfully inserted samples collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
158982025-04-30T06:44:27.491ZTRACtest_utilization (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal
158992025-04-30T06:44:27.491ZTRACtest_utilization (dns): buffer buffer = Length: 72 (0x48) bytes\n0000: 02 bc 01 00 00 01 00 00 00 00 00 01 09 5f 64 65 ............._de\n0010: 6e 64 72 69 74 65 04 5f 74 63 70 0d 63 6f 6e 74 ndrite._tcp.cont\n0020: 72 6f 6c 2d 70 6c 61 6e 65 05 6f 78 69 64 65 08 rol-plane.oxide.\n0030: 69 6e 74 65 72 6e 61 6c 00 00 21 00 01 00 00 29 internal..!....)\n0040: 04 d0 00 00 00 00 00 00 ........ kind = dns peer_addr = [::1]:50684 req_id = e0198770-f113-40c4-9d41-a66bc04fb1ca
159002025-04-30T06:44:27.491ZDEBGtest_utilization (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 700,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_dendrite._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_dendrite._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 9,\n 95,\n 100,\n 101,\n 110,\n 100,\n 114,\n 105,\n 116,\n 101,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:50684 req_id = e0198770-f113-40c4-9d41-a66bc04fb1ca
159012025-04-30T06:44:27.492ZDEBGtest_utilization (store): query key key = _dendrite._tcp
159022025-04-30T06:44:27.493ZDEBGtest_utilization (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
159032025-04-30T06:44:27.494ZDEBGtest_utilization (dns): dns response additional_records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal"), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] kind = dns peer_addr = [::1]:50684 query = LowerQuery { name: LowerName(Name("_dendrite._tcp.control-plane.oxide.internal.")), original: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN } } records = [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 47974, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal") })) }] req_id = e0198770-f113-40c4-9d41-a66bc04fb1ca
159042025-04-30T06:44:27.496ZDEBGtest_utilization (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 47974, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }], valid_until: Instant { tv_sec: 4332, tv_nsec: 295489606 } })
159052025-04-30T06:44:27.496ZTRACtest_utilization (DnsResolver): lookup_service_targets: looking up SRV target
159062025-04-30T06:44:27.496ZWARNtest_utilization (results-sink-cluster): failed to insert some results into metric DB collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 error = Failed to check out connection to database: No backends found for this service file = oximeter/collector/src/results_sink.rs:92
159072025-04-30T06:44:27.498ZTRACtest_utilization (dns): buffer buffer = Length: 108 (0x6c) bytes\n0000: da 0b 01 00 00 01 00 00 00 00 00 01 2d 64 65 6e ............-den\n0010: 64 72 69 74 65 2d 62 36 64 36 35 33 34 31 2d 31 drite-b6d65341-1\n0020: 36 37 63 2d 34 31 64 66 2d 39 62 35 63 2d 34 31 67c-41df-9b5c-41\n0030: 63 64 65 64 39 39 63 32 32 39 04 68 6f 73 74 0d cded99c229.host.\n0040: 63 6f 6e 74 72 6f 6c 2d 70 6c 61 6e 65 05 6f 78 control-plane.ox\n0050: 69 64 65 08 69 6e 74 65 72 6e 61 6c 00 00 1c 00 ide.internal....\n0060: 01 00 00 29 04 d0 00 00 00 00 00 00 ...)........ kind = dns peer_addr = [::1]:50705 req_id = bed33914-4fcc-4c22-9e06-450169999d5e
159082025-04-30T06:44:27.499ZDEBGtest_utilization (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 55819,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n query_type: AAAA,\n query_class: IN,\n },\n },\n original: [\n 45,\n 100,\n 101,\n 110,\n 100,\n 114,\n 105,\n 116,\n 101,\n 45,\n 98,\n 54,\n 100,\n 54,\n 53,\n 51,\n 52,\n 49,\n 45,\n 49,\n 54,\n 55,\n 99,\n 45,\n 52,\n 49,\n 100,\n 102,\n 45,\n 57,\n 98,\n 53,\n 99,\n 45,\n 52,\n 49,\n 99,\n 100,\n 101,\n 100,\n 57,\n 57,\n 99,\n 50,\n 50,\n 57,\n 4,\n 104,\n 111,\n 115,\n 116,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 28,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:50705 req_id = bed33914-4fcc-4c22-9e06-450169999d5e
159092025-04-30T06:44:27.499ZDEBGtest_utilization (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
159102025-04-30T06:44:27.499ZDEBGtest_utilization (dns): dns response additional_records = [] kind = dns peer_addr = [::1]:50705 query = LowerQuery { name: LowerName(Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.")), original: Query { name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), query_type: AAAA, query_class: IN } } records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] req_id = bed33914-4fcc-4c22-9e06-450169999d5e
159112025-04-30T06:44:27.499ZINFOtest_utilization (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager
159122025-04-30T06:44:27.579ZINFOtest_utilization (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager zone_address = ::1
159132025-04-30T06:44:27.579ZDEBGtest_utilization (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[::1]:12225/local/switch-id
159142025-04-30T06:44:27.579ZDEBGtest_utilization (MgsClient): client response background_task = bfd_manager result = Err(reqwest::Error { kind: Request, url: "http://[::1]:12225/local/switch-id", source: hyper_util::client::legacy::Error(Connect, ConnectError("tcp connect error", Os { code: 146, kind: ConnectionRefused, message: "Connection refused" })) })
159152025-04-30T06:44:27.579ZWARNtest_utilization (BackgroundTasks): failed to identify switch slot for dendrite background_task = bfd_manager reason = Communication Error: error sending request for url (http://[::1]:12225/local/switch-id): client error (Connect): tcp connect error: Connection refused (os error 146): Connection refused (os error 146) zone_address = ::1
159162025-04-30T06:44:27.581ZWARNtest_utilization (BackgroundTasks): Failed to map switch zone addr: Communication Error: error sending request for url (http://[::1]:12225/local/switch-id), retrying background_task = bfd_manager
159172025-04-30T06:44:27.584ZTRACtest_utilization (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal
159182025-04-30T06:44:27.597ZTRACtest_utilization (dns): buffer buffer = Length: 72 (0x48) bytes\n0000: f9 f4 01 00 00 01 00 00 00 00 00 01 09 5f 64 65 ............._de\n0010: 6e 64 72 69 74 65 04 5f 74 63 70 0d 63 6f 6e 74 ndrite._tcp.cont\n0020: 72 6f 6c 2d 70 6c 61 6e 65 05 6f 78 69 64 65 08 rol-plane.oxide.\n0030: 69 6e 74 65 72 6e 61 6c 00 00 21 00 01 00 00 29 internal..!....)\n0040: 04 d0 00 00 00 00 00 00 ........ kind = dns peer_addr = [::1]:52943 req_id = 509313bc-1def-4b55-940b-96f2d51f76f4
159192025-04-30T06:44:27.598ZDEBGtest_utilization (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 63988,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_dendrite._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_dendrite._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 9,\n 95,\n 100,\n 101,\n 110,\n 100,\n 114,\n 105,\n 116,\n 101,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:52943 req_id = 509313bc-1def-4b55-940b-96f2d51f76f4
159202025-04-30T06:44:27.599ZDEBGtest_utilization (store): query key key = _dendrite._tcp
159212025-04-30T06:44:27.599ZDEBGtest_utilization (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
159222025-04-30T06:44:27.599ZDEBGtest_utilization (dns): dns response additional_records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal"), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] kind = dns peer_addr = [::1]:52943 query = LowerQuery { name: LowerName(Name("_dendrite._tcp.control-plane.oxide.internal.")), original: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN } } records = [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 47974, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal") })) }] req_id = 509313bc-1def-4b55-940b-96f2d51f76f4
159232025-04-30T06:44:27.601ZDEBGtest_utilization (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 47974, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }], valid_until: Instant { tv_sec: 4332, tv_nsec: 400017541 } })
159242025-04-30T06:44:27.601ZTRACtest_utilization (DnsResolver): lookup_service_targets: looking up SRV target
159252025-04-30T06:44:27.629ZTRACtest_utilization (dns): buffer buffer = Length: 108 (0x6c) bytes\n0000: ae 0e 01 00 00 01 00 00 00 00 00 01 2d 64 65 6e ............-den\n0010: 64 72 69 74 65 2d 62 36 64 36 35 33 34 31 2d 31 drite-b6d65341-1\n0020: 36 37 63 2d 34 31 64 66 2d 39 62 35 63 2d 34 31 67c-41df-9b5c-41\n0030: 63 64 65 64 39 39 63 32 32 39 04 68 6f 73 74 0d cded99c229.host.\n0040: 63 6f 6e 74 72 6f 6c 2d 70 6c 61 6e 65 05 6f 78 control-plane.ox\n0050: 69 64 65 08 69 6e 74 65 72 6e 61 6c 00 00 1c 00 ide.internal....\n0060: 01 00 00 29 04 d0 00 00 00 00 00 00 ...)........ kind = dns peer_addr = [::1]:62712 req_id = bf05e233-f741-49b7-a289-9a2763ca48f7
159262025-04-30T06:44:27.629ZDEBGtest_utilization (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 44558,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n query_type: AAAA,\n query_class: IN,\n },\n },\n original: [\n 45,\n 100,\n 101,\n 110,\n 100,\n 114,\n 105,\n 116,\n 101,\n 45,\n 98,\n 54,\n 100,\n 54,\n 53,\n 51,\n 52,\n 49,\n 45,\n 49,\n 54,\n 55,\n 99,\n 45,\n 52,\n 49,\n 100,\n 102,\n 45,\n 57,\n 98,\n 53,\n 99,\n 45,\n 52,\n 49,\n 99,\n 100,\n 101,\n 100,\n 57,\n 57,\n 99,\n 50,\n 50,\n 57,\n 4,\n 104,\n 111,\n 115,\n 116,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 28,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:62712 req_id = bf05e233-f741-49b7-a289-9a2763ca48f7
159272025-04-30T06:44:27.629ZDEBGtest_utilization (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
159282025-04-30T06:44:27.629ZDEBGtest_utilization (dns): dns response additional_records = [] kind = dns peer_addr = [::1]:62712 query = LowerQuery { name: LowerName(Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.")), original: Query { name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), query_type: AAAA, query_class: IN } } records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] req_id = bf05e233-f741-49b7-a289-9a2763ca48f7
159292025-04-30T06:44:27.629ZWARNtest_utilization (datastore): Non-retryable transaction failure err = DatabaseError(UniqueViolation, "duplicate key value violates unique constraint \\"inv_caboose_pkey\\"") transaction = inventory_insert_collection
159302025-04-30T06:44:27.631ZWARNtest_utilization (BackgroundTasks): inventory collection failed background_task = inventory_collection error = failed to collect inventory: saving inventory to database: Internal Error: unexpected database error: duplicate key value violates unique constraint "inv_caboose_pkey"
159312025-04-30T06:44:27.631ZDEBGtest_utilization (BackgroundTasks): activation complete background_task = inventory_collection elapsed = 623.93026ms iteration = 45 status = {"error":"failed to collect inventory: saving inventory to database: Internal Error: unexpected database error: duplicate key value violates unique constraint \\"inv_caboose_pkey\\""}
159322025-04-30T06:44:27.632ZDEBGtest_utilization (BackgroundTasks): activating background_task = inventory_collection iteration = 46 reason = Signaled
159332025-04-30T06:44:27.632ZTRACtest_utilization (BackgroundTasks): authorize begin action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory
159342025-04-30T06:44:27.633ZTRACtest_utilization (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
159352025-04-30T06:44:27.633ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
159362025-04-30T06:44:27.633ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
159372025-04-30T06:44:27.634ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
159382025-04-30T06:44:27.651ZINFOtest_utilization (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager rack_id = c19a698f-c6f9-4a17-ae30-20d711b8f7dc
159392025-04-30T06:44:27.776ZINFOtest_utilization (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager rack_id = c19a698f-c6f9-4a17-ae30-20d711b8f7dc zone_address = ::1
159402025-04-30T06:44:27.776ZDEBGtest_utilization (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = c19a698f-c6f9-4a17-ae30-20d711b8f7dc uri = http://[::1]:12225/local/switch-id
159412025-04-30T06:44:27.776ZDEBGtest_utilization (MgsClient): client response background_task = switch_port_config_manager rack_id = c19a698f-c6f9-4a17-ae30-20d711b8f7dc result = Err(reqwest::Error { kind: Request, url: "http://[::1]:12225/local/switch-id", source: hyper_util::client::legacy::Error(Connect, ConnectError("tcp connect error", Os { code: 146, kind: ConnectionRefused, message: "Connection refused" })) })
159422025-04-30T06:44:27.776ZWARNtest_utilization (BackgroundTasks): failed to identify switch slot for dendrite background_task = switch_port_config_manager rack_id = c19a698f-c6f9-4a17-ae30-20d711b8f7dc reason = Communication Error: error sending request for url (http://[::1]:12225/local/switch-id): client error (Connect): tcp connect error: Connection refused (os error 146): Connection refused (os error 146) zone_address = ::1
159432025-04-30T06:44:27.778ZWARNtest_utilization (BackgroundTasks): Failed to map switch zone addr: Communication Error: error sending request for url (http://[::1]:12225/local/switch-id), retrying background_task = switch_port_config_manager rack_id = c19a698f-c6f9-4a17-ae30-20d711b8f7dc
159442025-04-30T06:44:27.782ZDEBGtest_utilization: attempt to fetch latest inventory collection result = Ok("found")
159452025-04-30T06:44:27.784ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
159462025-04-30T06:44:27.791ZDEBGtest_utilization (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory result = Ok(())
159472025-04-30T06:44:27.791ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
159482025-04-30T06:44:27.791ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
159492025-04-30T06:44:27.791ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
159502025-04-30T06:44:27.804ZDEBGtest_utilization (BackgroundTasks): inventory_prune_one: nothing eligible for removal (too few) background_task = inventory_collection candidates = [(13d8b8d9-6c6b-47a6-9190-952544edb75b, 0)]
159512025-04-30T06:44:27.804ZTRACtest_utilization (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal
159522025-04-30T06:44:27.813ZTRACtest_utilization (dns): buffer buffer = Length: 67 (0x43) bytes\n0000: c2 5c 01 00 00 01 00 00 00 00 00 01 04 5f 6d 67 .\\..........._mg\n0010: 73 04 5f 74 63 70 0d 63 6f 6e 74 72 6f 6c 2d 70 s._tcp.control-p\n0020: 6c 61 6e 65 05 6f 78 69 64 65 08 69 6e 74 65 72 lane.oxide.inter\n0030: 6e 61 6c 00 00 21 00 01 00 00 29 04 d0 00 00 00 nal..!....).....\n0040: 00 00 00 ... kind = dns peer_addr = [::1]:60223 req_id = df95ffae-7064-42a6-a027-b6276e0e6fe5
159532025-04-30T06:44:27.813ZDEBGtest_utilization (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 49756,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_mgs._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_mgs._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 4,\n 95,\n 109,\n 103,\n 115,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:60223 req_id = df95ffae-7064-42a6-a027-b6276e0e6fe5
159542025-04-30T06:44:27.814ZDEBGtest_utilization (store): query key key = _mgs._tcp
159552025-04-30T06:44:27.814ZDEBGtest_utilization (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
159562025-04-30T06:44:27.815ZDEBGtest_utilization (dns): dns response additional_records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal"), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] kind = dns peer_addr = [::1]:60223 query = LowerQuery { name: LowerName(Name("_mgs._tcp.control-plane.oxide.internal.")), original: Query { name: Name("_mgs._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN } } records = [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 48759, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal") })) }] req_id = df95ffae-7064-42a6-a027-b6276e0e6fe5
159572025-04-30T06:44:27.816ZDEBGtest_utilization (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgs._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 48759, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }], valid_until: Instant { tv_sec: 4332, tv_nsec: 615476980 } })
159582025-04-30T06:44:27.816ZTRACtest_utilization (DnsResolver): lookup_service_targets: looking up SRV target
159592025-04-30T06:44:27.820ZTRACtest_utilization (dns): buffer buffer = Length: 108 (0x6c) bytes\n0000: 7f 38 01 00 00 01 00 00 00 00 00 01 2d 64 65 6e .8..........-den\n0010: 64 72 69 74 65 2d 62 36 64 36 35 33 34 31 2d 31 drite-b6d65341-1\n0020: 36 37 63 2d 34 31 64 66 2d 39 62 35 63 2d 34 31 67c-41df-9b5c-41\n0030: 63 64 65 64 39 39 63 32 32 39 04 68 6f 73 74 0d cded99c229.host.\n0040: 63 6f 6e 74 72 6f 6c 2d 70 6c 61 6e 65 05 6f 78 control-plane.ox\n0050: 69 64 65 08 69 6e 74 65 72 6e 61 6c 00 00 1c 00 ide.internal....\n0060: 01 00 00 29 04 d0 00 00 00 00 00 00 ...)........ kind = dns peer_addr = [::1]:56080 req_id = 94385819-27f6-4a82-a35f-5349a6ad085a
159602025-04-30T06:44:27.820ZDEBGtest_utilization (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 32568,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n query_type: AAAA,\n query_class: IN,\n },\n },\n original: [\n 45,\n 100,\n 101,\n 110,\n 100,\n 114,\n 105,\n 116,\n 101,\n 45,\n 98,\n 54,\n 100,\n 54,\n 53,\n 51,\n 52,\n 49,\n 45,\n 49,\n 54,\n 55,\n 99,\n 45,\n 52,\n 49,\n 100,\n 102,\n 45,\n 57,\n 98,\n 53,\n 99,\n 45,\n 52,\n 49,\n 99,\n 100,\n 101,\n 100,\n 57,\n 57,\n 99,\n 50,\n 50,\n 57,\n 4,\n 104,\n 111,\n 115,\n 116,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 28,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:56080 req_id = 94385819-27f6-4a82-a35f-5349a6ad085a
159612025-04-30T06:44:27.820ZDEBGtest_utilization (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
159622025-04-30T06:44:27.820ZDEBGtest_utilization (dns): dns response additional_records = [] kind = dns peer_addr = [::1]:56080 query = LowerQuery { name: LowerName(Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.")), original: Query { name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), query_type: AAAA, query_class: IN } } records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] req_id = 94385819-27f6-4a82-a35f-5349a6ad085a
159632025-04-30T06:44:27.898ZTRACtest_utilization (DnsResolver): lookup_all_socket_v6 srv dns_name = _clickhouse-admin-keeper._tcp.control-plane.oxide.internal
159642025-04-30T06:44:27.909ZTRACtest_utilization (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory
159652025-04-30T06:44:27.909ZTRACtest_utilization (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
159662025-04-30T06:44:27.909ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
159672025-04-30T06:44:27.910ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
159682025-04-30T06:44:27.911ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
159692025-04-30T06:44:27.914ZTRACtest_utilization (dns): buffer buffer = Length: 87 (0x57) bytes\n0000: 25 2e 01 00 00 01 00 00 00 00 00 01 18 5f 63 6c %............_cl\n0010: 69 63 6b 68 6f 75 73 65 2d 61 64 6d 69 6e 2d 6b ickhouse-admin-k\n0020: 65 65 70 65 72 04 5f 74 63 70 0d 63 6f 6e 74 72 eeper._tcp.contr\n0030: 6f 6c 2d 70 6c 61 6e 65 05 6f 78 69 64 65 08 69 ol-plane.oxide.i\n0040: 6e 74 65 72 6e 61 6c 00 00 21 00 01 00 00 29 04 nternal..!....).\n0050: d0 00 00 00 00 00 00 ....... kind = dns peer_addr = [::1]:65388 req_id = 9b993a64-3a97-4ba6-a14f-b6f197d6dbd0
159702025-04-30T06:44:27.915ZDEBGtest_utilization (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 9518,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 24,\n 95,\n 99,\n 108,\n 105,\n 99,\n 107,\n 104,\n 111,\n 117,\n 115,\n 101,\n 45,\n 97,\n 100,\n 109,\n 105,\n 110,\n 45,\n 107,\n 101,\n 101,\n 112,\n 101,\n 114,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:65388 req_id = 9b993a64-3a97-4ba6-a14f-b6f197d6dbd0
159712025-04-30T06:44:27.915ZDEBGtest_utilization (store): query key key = _clickhouse-admin-keeper._tcp
159722025-04-30T06:44:27.915ZERROtest_utilization (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 9518, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 1, }, query: WireQuery { query: LowerQuery { name: LowerName( Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), ), original: Query { name: Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN, }, }, original: [ 24, 95, 99, 108, 105, 99, 107, 104, 111, 117, 115, 101, 45, 97, 100, 109, 105, 110, 45, 107, 101, 101, 112, 101, 114, 4, 95, 116, 99, 112, 13, 99, 111, 110, 116, 114, 111, 108, 45, 112, 108, 97, 110, 101, 5, 111, 120, 105, 100, 101, 8, 105, 110, 116, 101, 114, 110, 97, 108, 0, 0, 33, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: Some( Edns { rcode_high: 0, version: 0, dnssec_ok: false, max_payload: 1232, options: OPT { options: {}, }, }, ), } NXDOMAIN: no records found for name: "_clickhouse-admin-keeper._tcp.control-plane.oxide.internal." kind = dns peer_addr = [::1]:65388 req_id = 9b993a64-3a97-4ba6-a14f-b6f197d6dbd0
159732025-04-30T06:44:27.918ZDEBGtest_utilization (BackgroundTasks): begin collection background_task = inventory_collection
159742025-04-30T06:44:27.918ZDEBGtest_utilization (BackgroundTasks): begin collection from MGS background_task = inventory_collection mgs_url = http://[::1]:48759
159752025-04-30T06:44:27.918ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/ignition
159762025-04-30T06:44:27.919ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
159772025-04-30T06:44:27.927ZDEBGtest_utilization (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory result = Ok(())
159782025-04-30T06:44:27.927ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
159792025-04-30T06:44:27.928ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
159802025-04-30T06:44:27.940ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
159812025-04-30T06:44:27.945ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
159822025-04-30T06:44:27.946ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
159832025-04-30T06:44:27.946ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
159842025-04-30T06:44:27.948ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/ignition", status: 200, headers: {"content-type": "application/json", "x-request-id": "ab0a0dc1-33f9-4e87-a7b8-19d72289fe44", "content-length": "835", "date": "Wed, 30 Apr 2025 06:44:27 GMT"} })
159852025-04-30T06:44:27.950ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0
159862025-04-30T06:44:27.961ZINFOtest_utilization (oximeter-agent): refreshing list of producers from Nexus collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 file = oximeter/collector/src/agent.rs:562
159872025-04-30T06:44:27.962ZDEBGtest_utilization (oximeter): client request body = None method = GET uri = http://127.0.0.1:32956/metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?sort_by=id_ascending
159882025-04-30T06:44:27.964ZTRACtest_utilization (dropshot_internal): incoming request local_addr = 127.0.0.1:32956 method = GET remote_addr = 127.0.0.1:55797 req_id = a1ec0f91-4dc5-47cd-b135-c83d159cb8fe uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?sort_by=id_ascending
159892025-04-30T06:44:27.964ZTRACtest_utilization (dropshot_internal): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = 127.0.0.1:32956 method = GET remote_addr = 127.0.0.1:55797 req_id = a1ec0f91-4dc5-47cd-b135-c83d159cb8fe resource = Database uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?sort_by=id_ascending
159902025-04-30T06:44:27.964ZDEBGtest_utilization (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = 127.0.0.1:32956 method = GET remote_addr = 127.0.0.1:55797 req_id = a1ec0f91-4dc5-47cd-b135-c83d159cb8fe roles = RoleSet { roles: {} } uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?sort_by=id_ascending
159912025-04-30T06:44:27.965ZDEBGtest_utilization (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = 127.0.0.1:32956 method = GET remote_addr = 127.0.0.1:55797 req_id = a1ec0f91-4dc5-47cd-b135-c83d159cb8fe resource = Database result = Ok(()) uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?sort_by=id_ascending
159922025-04-30T06:44:27.970ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "23e237a2-c607-4061-9a43-e82794c377a1", "content-length": "734", "date": "Wed, 30 Apr 2025 06:44:27 GMT"} })
159932025-04-30T06:44:27.970ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0/component/sp/caboose?firmware_slot=0
159942025-04-30T06:44:27.971ZINFOtest_utilization (dropshot_internal): request completed latency_us = 6873 local_addr = 127.0.0.1:32956 method = GET remote_addr = 127.0.0.1:55797 req_id = a1ec0f91-4dc5-47cd-b135-c83d159cb8fe response_code = 200 uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?sort_by=id_ascending
159952025-04-30T06:44:27.971ZDEBGtest_utilization (oximeter): client response result = Ok(Response { url: "http://127.0.0.1:32956/metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?sort_by=id_ascending", status: 200, headers: {"content-type": "application/json", "x-request-id": "a1ec0f91-4dc5-47cd-b135-c83d159cb8fe", "content-length": "405", "date": "Wed, 30 Apr 2025 06:44:27 GMT"} })
159962025-04-30T06:44:27.971ZDEBGtest_utilization (oximeter): client request body = None method = GET uri = http://127.0.0.1:32956/metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJhYzU1ZDhkOS1hMjk4LTQ1MzQtODk4Yi1iZWVlMWM5ZTRlNTIifX0%3D
159972025-04-30T06:44:27.972ZTRACtest_utilization (dropshot_internal): incoming request local_addr = 127.0.0.1:32956 method = GET remote_addr = 127.0.0.1:55797 req_id = c35daf00-a6b9-4f26-928b-c8511033ece6 uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJhYzU1ZDhkOS1hMjk4LTQ1MzQtODk4Yi1iZWVlMWM5ZTRlNTIifX0%3D
159982025-04-30T06:44:27.972ZTRACtest_utilization (dropshot_internal): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = 127.0.0.1:32956 method = GET remote_addr = 127.0.0.1:55797 req_id = c35daf00-a6b9-4f26-928b-c8511033ece6 resource = Database uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJhYzU1ZDhkOS1hMjk4LTQ1MzQtODk4Yi1iZWVlMWM5ZTRlNTIifX0%3D
159992025-04-30T06:44:27.972ZDEBGtest_utilization (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = 127.0.0.1:32956 method = GET remote_addr = 127.0.0.1:55797 req_id = c35daf00-a6b9-4f26-928b-c8511033ece6 roles = RoleSet { roles: {} } uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJhYzU1ZDhkOS1hMjk4LTQ1MzQtODk4Yi1iZWVlMWM5ZTRlNTIifX0%3D
160002025-04-30T06:44:27.973ZDEBGtest_utilization (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = 127.0.0.1:32956 method = GET remote_addr = 127.0.0.1:55797 req_id = c35daf00-a6b9-4f26-928b-c8511033ece6 resource = Database result = Ok(()) uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJhYzU1ZDhkOS1hMjk4LTQ1MzQtODk4Yi1iZWVlMWM5ZTRlNTIifX0%3D
160012025-04-30T06:44:27.980ZINFOtest_utilization (dropshot_internal): request completed latency_us = 7821 local_addr = 127.0.0.1:32956 method = GET remote_addr = 127.0.0.1:55797 req_id = c35daf00-a6b9-4f26-928b-c8511033ece6 response_code = 200 uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJhYzU1ZDhkOS1hMjk4LTQ1MzQtODk4Yi1iZWVlMWM5ZTRlNTIifX0%3D
160022025-04-30T06:44:27.981ZDEBGtest_utilization (oximeter): client response result = Ok(Response { url: "http://127.0.0.1:32956/metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJhYzU1ZDhkOS1hMjk4LTQ1MzQtODk4Yi1iZWVlMWM5ZTRlNTIifX0%3D", status: 200, headers: {"content-type": "application/json", "x-request-id": "c35daf00-a6b9-4f26-928b-c8511033ece6", "content-length": "29", "date": "Wed, 30 Apr 2025 06:44:27 GMT"} })
160032025-04-30T06:44:27.981ZTRACtest_utilization (oximeter-agent): ignoring request to update existing metric producer, since the endpoint information is the same as the existing collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
160042025-04-30T06:44:27.981ZTRACtest_utilization (oximeter-agent): ignoring request to update existing metric producer, since the endpoint information is the same as the existing collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = ac55d8d9-a298-4534-898b-beee1c9e4e52
160052025-04-30T06:44:27.981ZINFOtest_utilization (oximeter-agent): refreshed list of producers from Nexus collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 file = oximeter/collector/src/agent.rs:637 n_current_tasks = 2 n_pruned_tasks = 0
160062025-04-30T06:44:27.981ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "0d9f27db-7a0c-4f04-84f1-0dba305c2e00", "content-length": "111", "date": "Wed, 30 Apr 2025 06:44:27 GMT"} })
160072025-04-30T06:44:27.981ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0/component/sp/caboose?firmware_slot=1
160082025-04-30T06:44:27.984ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "7c7de429-f77a-40bb-b103-7c7816bc9a39", "content-length": "111", "date": "Wed, 30 Apr 2025 06:44:27 GMT"} })
160092025-04-30T06:44:27.984ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0/component/rot/caboose?firmware_slot=0
160102025-04-30T06:44:27.987ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "627b6212-cc6d-4290-8181-3a21efa97ec7", "content-length": "170", "date": "Wed, 30 Apr 2025 06:44:27 GMT"} })
160112025-04-30T06:44:27.988ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0/component/rot/caboose?firmware_slot=1
160122025-04-30T06:44:27.990ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "b2b50dbf-c0c3-4e0c-a0d0-9ef053476434", "content-length": "170", "date": "Wed, 30 Apr 2025 06:44:27 GMT"} })
160132025-04-30T06:44:27.991ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0/component/stage0/caboose?firmware_slot=0
160142025-04-30T06:44:27.994ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "21ed4e91-bab1-4a9d-96dc-27f520cfc082", "content-length": "179", "date": "Wed, 30 Apr 2025 06:44:27 GMT"} })
160152025-04-30T06:44:27.994ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0/component/stage0/caboose?firmware_slot=1
160162025-04-30T06:44:27.997ZDEBGtest_utilization (collection-task): sent timer-based collection request to the collection task collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
160172025-04-30T06:44:27.997ZDEBGtest_utilization (collection-task): collecting from producer collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
160182025-04-30T06:44:27.999ZDEBGtest_utilization: attempt to fetch latest inventory collection result = Ok("found")
160192025-04-30T06:44:27.999ZTRACtest_utilization (collection-task): sent collection request to producer collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
160202025-04-30T06:44:28.004ZDEBGtest_utilization (collection-task): collected results from producer collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 n_results = 1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
160212025-04-30T06:44:28.004ZTRACtest_utilization (collection-task): forwarded results to main collection loop collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
160222025-04-30T06:44:28.004ZTRACtest_utilization (collection-task): top of inner collection loop, waiting for next request collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
160232025-04-30T06:44:28.004ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "a2120bf1-2e60-40c6-a8b3-4de8198a2009", "content-length": "179", "date": "Wed, 30 Apr 2025 06:44:27 GMT"} })
160242025-04-30T06:44:28.005ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0/component/rot/cmpa
160252025-04-30T06:44:28.005ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "31acf016-dccc-43e0-9181-78ae656a2c80", "content-length": "702", "date": "Wed, 30 Apr 2025 06:44:27 GMT"} })
160262025-04-30T06:44:28.005ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0/component/rot/cfpa
160272025-04-30T06:44:28.006ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "2b67a52b-abe9-4979-9da6-1de314f0fe80", "content-length": "727", "date": "Wed, 30 Apr 2025 06:44:27 GMT"} })
160282025-04-30T06:44:28.006ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0/component/rot/cfpa
160292025-04-30T06:44:28.007ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "62bdaef7-413f-4c04-9094-7f4e01a6b0d5", "content-length": "729", "date": "Wed, 30 Apr 2025 06:44:27 GMT"} })
160302025-04-30T06:44:28.007ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0/component/rot/cfpa
160312025-04-30T06:44:28.009ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "264c7491-5d8b-4466-8d83-4910f653b598", "content-length": "728", "date": "Wed, 30 Apr 2025 06:44:27 GMT"} })
160322025-04-30T06:44:28.009ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1
160332025-04-30T06:44:28.010ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1", status: 200, headers: {"content-type": "application/json", "x-request-id": "39b7f1f4-339d-4f70-8741-464f72da9167", "content-length": "734", "date": "Wed, 30 Apr 2025 06:44:27 GMT"} })
160342025-04-30T06:44:28.011ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1/component/sp/caboose?firmware_slot=0
160352025-04-30T06:44:28.013ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "8b5cd7c3-d2e4-40c3-987c-3b57d7f206d4", "content-length": "111", "date": "Wed, 30 Apr 2025 06:44:27 GMT"} })
160362025-04-30T06:44:28.013ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1/component/sp/caboose?firmware_slot=1
160372025-04-30T06:44:28.016ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "2374803c-11f5-4217-a6fc-262f55ab96a6", "content-length": "111", "date": "Wed, 30 Apr 2025 06:44:27 GMT"} })
160382025-04-30T06:44:28.016ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1/component/rot/caboose?firmware_slot=0
160392025-04-30T06:44:28.018ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "a0467ad7-db9c-4700-a2d1-57ed1f93246e", "content-length": "170", "date": "Wed, 30 Apr 2025 06:44:27 GMT"} })
160402025-04-30T06:44:28.018ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1/component/rot/caboose?firmware_slot=1
160412025-04-30T06:44:28.021ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "27b2b2f5-18c3-4ce5-99a1-628373b05660", "content-length": "170", "date": "Wed, 30 Apr 2025 06:44:27 GMT"} })
160422025-04-30T06:44:28.021ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1/component/stage0/caboose?firmware_slot=0
160432025-04-30T06:44:28.024ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "84c00d7b-7693-4142-9791-5ea8ca9fca1f", "content-length": "179", "date": "Wed, 30 Apr 2025 06:44:27 GMT"} })
160442025-04-30T06:44:28.024ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1/component/stage0/caboose?firmware_slot=1
160452025-04-30T06:44:28.028ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "b2f321d0-e35b-47b5-b3a3-4aab9c68803b", "content-length": "179", "date": "Wed, 30 Apr 2025 06:44:27 GMT"} })
160462025-04-30T06:44:28.028ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1/component/rot/cmpa
160472025-04-30T06:44:28.030ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "1303a8d7-a4f6-4c32-b0df-fe7afaef3b4b", "content-length": "702", "date": "Wed, 30 Apr 2025 06:44:27 GMT"} })
160482025-04-30T06:44:28.030ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1/component/rot/cfpa
160492025-04-30T06:44:28.032ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "a29042cd-ce93-4d91-a7fc-5559d1528a9d", "content-length": "727", "date": "Wed, 30 Apr 2025 06:44:27 GMT"} })
160502025-04-30T06:44:28.032ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1/component/rot/cfpa
160512025-04-30T06:44:28.034ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "03c2adfc-389b-4a64-be35-5ee3442c5eb1", "content-length": "729", "date": "Wed, 30 Apr 2025 06:44:27 GMT"} })
160522025-04-30T06:44:28.034ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1/component/rot/cfpa
160532025-04-30T06:44:28.036ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "543a5ddf-15ac-437b-9fe0-b1ea61fd17a8", "content-length": "728", "date": "Wed, 30 Apr 2025 06:44:27 GMT"} })
160542025-04-30T06:44:28.036ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0
160552025-04-30T06:44:28.039ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "62026c44-47dd-4635-ac93-9b786c65912e", "content-length": "723", "date": "Wed, 30 Apr 2025 06:44:27 GMT"} })
160562025-04-30T06:44:28.039ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0/component/sp/caboose?firmware_slot=0
160572025-04-30T06:44:28.043ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "e5c0bcd5-0aea-4391-b91a-99dfa85ffd25", "content-length": "109", "date": "Wed, 30 Apr 2025 06:44:27 GMT"} })
160582025-04-30T06:44:28.043ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0/component/sp/caboose?firmware_slot=1
160592025-04-30T06:44:28.048ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "11249caa-b4a5-45d0-8106-7c196aed9b3b", "content-length": "109", "date": "Wed, 30 Apr 2025 06:44:27 GMT"} })
160602025-04-30T06:44:28.048ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0/component/rot/caboose?firmware_slot=0
160612025-04-30T06:44:28.050ZTRACtest_utilization (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory
160622025-04-30T06:44:28.050ZTRACtest_utilization (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
160632025-04-30T06:44:28.050ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
160642025-04-30T06:44:28.050ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
160652025-04-30T06:44:28.051ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
160662025-04-30T06:44:28.054ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
160672025-04-30T06:44:28.060ZDEBGtest_utilization (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory result = Ok(())
160682025-04-30T06:44:28.060ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
160692025-04-30T06:44:28.060ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
160702025-04-30T06:44:28.061ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
160712025-04-30T06:44:28.064ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "0646546c-2e1a-4d4e-bffe-e534261cf6b4", "content-length": "169", "date": "Wed, 30 Apr 2025 06:44:27 GMT"} })
160722025-04-30T06:44:28.064ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0/component/rot/caboose?firmware_slot=1
160732025-04-30T06:44:28.065ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
160742025-04-30T06:44:28.065ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
160752025-04-30T06:44:28.068ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
160762025-04-30T06:44:28.082ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "065e4824-b8a3-44ab-8dbb-4417f03c4463", "content-length": "169", "date": "Wed, 30 Apr 2025 06:44:27 GMT"} })
160772025-04-30T06:44:28.082ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0/component/stage0/caboose?firmware_slot=0
160782025-04-30T06:44:28.086ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "32326afa-6d8e-49cb-9c5b-98d381f59a5d", "content-length": "178", "date": "Wed, 30 Apr 2025 06:44:27 GMT"} })
160792025-04-30T06:44:28.086ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0/component/stage0/caboose?firmware_slot=1
160802025-04-30T06:44:28.089ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "374376ed-c256-44ad-b8d9-bbe0d5a0783b", "content-length": "178", "date": "Wed, 30 Apr 2025 06:44:27 GMT"} })
160812025-04-30T06:44:28.089ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0/component/rot/cmpa
160822025-04-30T06:44:28.091ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "7fe02954-6e6f-4529-9229-2f43f9786bce", "content-length": "702", "date": "Wed, 30 Apr 2025 06:44:27 GMT"} })
160832025-04-30T06:44:28.092ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0/component/rot/cfpa
160842025-04-30T06:44:28.093ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "32342002-dfa9-4073-880e-8d26c9b8873f", "content-length": "727", "date": "Wed, 30 Apr 2025 06:44:27 GMT"} })
160852025-04-30T06:44:28.093ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0/component/rot/cfpa
160862025-04-30T06:44:28.095ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "5eecc057-1b5c-4279-9954-1146dc8c7a08", "content-length": "729", "date": "Wed, 30 Apr 2025 06:44:27 GMT"} })
160872025-04-30T06:44:28.096ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0/component/rot/cfpa
160882025-04-30T06:44:28.099ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "047a597a-2651-475c-b0a5-7289fec87f95", "content-length": "728", "date": "Wed, 30 Apr 2025 06:44:27 GMT"} })
160892025-04-30T06:44:28.099ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1
160902025-04-30T06:44:28.103ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1", status: 200, headers: {"content-type": "application/json", "x-request-id": "5a56104f-4949-4c4a-b65c-928b47456520", "content-length": "723", "date": "Wed, 30 Apr 2025 06:44:27 GMT"} })
160912025-04-30T06:44:28.103ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1/component/sp/caboose?firmware_slot=0
160922025-04-30T06:44:28.106ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "c41f1dd8-de90-4117-afe8-6946bfaff886", "content-length": "109", "date": "Wed, 30 Apr 2025 06:44:27 GMT"} })
160932025-04-30T06:44:28.106ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1/component/sp/caboose?firmware_slot=1
160942025-04-30T06:44:28.116ZDEBGtest_utilization: attempt to fetch latest inventory collection result = Ok("found")
160952025-04-30T06:44:28.117ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "286b4bc0-1688-486a-86c4-824b63cd8941", "content-length": "109", "date": "Wed, 30 Apr 2025 06:44:27 GMT"} })
160962025-04-30T06:44:28.117ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1/component/rot/caboose?firmware_slot=0
160972025-04-30T06:44:28.120ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "7cdcf23d-c8f1-4ab2-be9f-8f24206937ba", "content-length": "169", "date": "Wed, 30 Apr 2025 06:44:27 GMT"} })
160982025-04-30T06:44:28.121ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1/component/rot/caboose?firmware_slot=1
160992025-04-30T06:44:28.122ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "aca10716-19ad-448e-981c-e9307e685810", "content-length": "169", "date": "Wed, 30 Apr 2025 06:44:27 GMT"} })
161002025-04-30T06:44:28.123ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1/component/stage0/caboose?firmware_slot=0
161012025-04-30T06:44:28.124ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "8d1d13bb-9d59-40df-809e-94641d47faf3", "content-length": "178", "date": "Wed, 30 Apr 2025 06:44:27 GMT"} })
161022025-04-30T06:44:28.124ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1/component/stage0/caboose?firmware_slot=1
161032025-04-30T06:44:28.127ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "da2e1224-eb9d-4e77-8efd-ded271c8cbf0", "content-length": "178", "date": "Wed, 30 Apr 2025 06:44:27 GMT"} })
161042025-04-30T06:44:28.127ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1/component/rot/cmpa
161052025-04-30T06:44:28.128ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "8b297b49-c3a5-47b8-be03-40b95dbf302e", "content-length": "702", "date": "Wed, 30 Apr 2025 06:44:27 GMT"} })
161062025-04-30T06:44:28.128ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1/component/rot/cfpa
161072025-04-30T06:44:28.130ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "2b479b28-c59a-42e1-8d72-263316831881", "content-length": "727", "date": "Wed, 30 Apr 2025 06:44:27 GMT"} })
161082025-04-30T06:44:28.130ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1/component/rot/cfpa
161092025-04-30T06:44:28.131ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "32002d08-5a4b-4d7d-8524-8d26ab1ac750", "content-length": "729", "date": "Wed, 30 Apr 2025 06:44:27 GMT"} })
161102025-04-30T06:44:28.131ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1/component/rot/cfpa
161112025-04-30T06:44:28.132ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "0954bc6f-9898-44b6-ac07-a56d92e48016", "content-length": "728", "date": "Wed, 30 Apr 2025 06:44:27 GMT"} })
161122025-04-30T06:44:28.132ZTRACtest_utilization (BackgroundTasks): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet
161132025-04-30T06:44:28.132ZTRACtest_utilization (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
161142025-04-30T06:44:28.132ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
161152025-04-30T06:44:28.133ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
161162025-04-30T06:44:28.134ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
161172025-04-30T06:44:28.139ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
161182025-04-30T06:44:28.154ZDEBGtest_utilization (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet result = Ok(())
161192025-04-30T06:44:28.154ZTRACtest_utilization (BackgroundTasks): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet
161202025-04-30T06:44:28.154ZTRACtest_utilization (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
161212025-04-30T06:44:28.154ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
161222025-04-30T06:44:28.154ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
161232025-04-30T06:44:28.155ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
161242025-04-30T06:44:28.170ZTRACtest_utilization (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory
161252025-04-30T06:44:28.170ZTRACtest_utilization (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
161262025-04-30T06:44:28.170ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
161272025-04-30T06:44:28.170ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
161282025-04-30T06:44:28.171ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
161292025-04-30T06:44:28.178ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
161302025-04-30T06:44:28.184ZDEBGtest_utilization (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet result = Ok(())
161312025-04-30T06:44:28.184ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
161322025-04-30T06:44:28.184ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
161332025-04-30T06:44:28.186ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
161342025-04-30T06:44:28.186ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
161352025-04-30T06:44:28.191ZDEBGtest_utilization (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory result = Ok(())
161362025-04-30T06:44:28.191ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
161372025-04-30T06:44:28.191ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
161382025-04-30T06:44:28.192ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
161392025-04-30T06:44:28.197ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
161402025-04-30T06:44:28.197ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
161412025-04-30T06:44:28.197ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
161422025-04-30T06:44:28.249ZDEBGtest_utilization (BackgroundTasks): begin collection from Sled Agent background_task = inventory_collection sled_agent_url = http://[::1]:39365
161432025-04-30T06:44:28.250ZDEBGtest_utilization (BackgroundTasks): client request SledAgent = http://[::1]:39365 background_task = inventory_collection body = None method = GET uri = http://[::1]:39365/inventory
161442025-04-30T06:44:28.251ZINFOtest_utilization (dropshot): accepted connection local_addr = [::1]:39365 remote_addr = [::1]:60423 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
161452025-04-30T06:44:28.251ZTRACtest_utilization (dropshot): incoming request local_addr = [::1]:39365 method = GET remote_addr = [::1]:60423 req_id = 59b58432-539c-4a26-a592-6b19a40b693d sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /inventory
161462025-04-30T06:44:28.252ZINFOtest_utilization (dropshot): request completed latency_us = 533 local_addr = [::1]:39365 method = GET remote_addr = [::1]:60423 req_id = 59b58432-539c-4a26-a592-6b19a40b693d response_code = 200 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /inventory
161472025-04-30T06:44:28.252ZDEBGtest_utilization (BackgroundTasks): client response SledAgent = http://[::1]:39365 background_task = inventory_collection result = Ok(Response { url: "http://[::1]:39365/inventory", status: 200, headers: {"content-type": "application/json", "x-request-id": "59b58432-539c-4a26-a592-6b19a40b693d", "content-length": "3141", "date": "Wed, 30 Apr 2025 06:44:27 GMT"} })
161482025-04-30T06:44:28.256ZDEBGtest_utilization (BackgroundTasks): begin collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
161492025-04-30T06:44:28.256ZDEBGtest_utilization (BackgroundTasks): end collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
161502025-04-30T06:44:28.256ZDEBGtest_utilization (BackgroundTasks): finished collection background_task = inventory_collection
161512025-04-30T06:44:28.260ZTRACtest_utilization (BackgroundTasks): authorize begin action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory
161522025-04-30T06:44:28.260ZTRACtest_utilization (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
161532025-04-30T06:44:28.260ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
161542025-04-30T06:44:28.261ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
161552025-04-30T06:44:28.262ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
161562025-04-30T06:44:28.268ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
161572025-04-30T06:44:28.274ZDEBGtest_utilization (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory result = Ok(())
161582025-04-30T06:44:28.274ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
161592025-04-30T06:44:28.274ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
161602025-04-30T06:44:28.274ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
161612025-04-30T06:44:28.315ZDEBGtest_utilization: attempt to fetch latest inventory collection result = Ok("found")
161622025-04-30T06:44:28.322ZWARNtest_utilization (datastore): Non-retryable transaction failure err = DatabaseError(UniqueViolation, "duplicate key value violates unique constraint \\"inv_caboose_pkey\\"") transaction = inventory_insert_collection
161632025-04-30T06:44:28.323ZWARNtest_utilization (BackgroundTasks): inventory collection failed background_task = inventory_collection error = failed to collect inventory: saving inventory to database: Internal Error: unexpected database error: duplicate key value violates unique constraint "inv_caboose_pkey"
161642025-04-30T06:44:28.323ZDEBGtest_utilization (BackgroundTasks): activation complete background_task = inventory_collection elapsed = 690.853721ms iteration = 46 status = {"error":"failed to collect inventory: saving inventory to database: Internal Error: unexpected database error: duplicate key value violates unique constraint \\"inv_caboose_pkey\\""}
161652025-04-30T06:44:28.323ZDEBGtest_utilization (BackgroundTasks): activating background_task = inventory_collection iteration = 47 reason = Signaled
161662025-04-30T06:44:28.323ZTRACtest_utilization (BackgroundTasks): authorize begin action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory
161672025-04-30T06:44:28.323ZTRACtest_utilization (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
161682025-04-30T06:44:28.323ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
161692025-04-30T06:44:28.323ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
161702025-04-30T06:44:28.324ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
161712025-04-30T06:44:28.328ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
161722025-04-30T06:44:28.333ZDEBGtest_utilization (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory result = Ok(())
161732025-04-30T06:44:28.333ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
161742025-04-30T06:44:28.333ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
161752025-04-30T06:44:28.333ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
161762025-04-30T06:44:28.339ZDEBGtest_utilization (BackgroundTasks): inventory_prune_one: nothing eligible for removal (too few) background_task = inventory_collection candidates = [(13d8b8d9-6c6b-47a6-9190-952544edb75b, 0)]
161772025-04-30T06:44:28.339ZTRACtest_utilization (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal
161782025-04-30T06:44:28.340ZTRACtest_utilization (dns): buffer buffer = Length: 67 (0x43) bytes\n0000: bc 34 01 00 00 01 00 00 00 00 00 01 04 5f 6d 67 .4..........._mg\n0010: 73 04 5f 74 63 70 0d 63 6f 6e 74 72 6f 6c 2d 70 s._tcp.control-p\n0020: 6c 61 6e 65 05 6f 78 69 64 65 08 69 6e 74 65 72 lane.oxide.inter\n0030: 6e 61 6c 00 00 21 00 01 00 00 29 04 d0 00 00 00 nal..!....).....\n0040: 00 00 00 ... kind = dns peer_addr = [::1]:58945 req_id = 4c29747a-5370-4c4d-a7b6-7e0a06e55440
161792025-04-30T06:44:28.340ZDEBGtest_utilization (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 48180,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_mgs._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_mgs._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 4,\n 95,\n 109,\n 103,\n 115,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:58945 req_id = 4c29747a-5370-4c4d-a7b6-7e0a06e55440
161802025-04-30T06:44:28.340ZDEBGtest_utilization (store): query key key = _mgs._tcp
161812025-04-30T06:44:28.340ZDEBGtest_utilization (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
161822025-04-30T06:44:28.340ZDEBGtest_utilization (dns): dns response additional_records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal"), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] kind = dns peer_addr = [::1]:58945 query = LowerQuery { name: LowerName(Name("_mgs._tcp.control-plane.oxide.internal.")), original: Query { name: Name("_mgs._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN } } records = [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 48759, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal") })) }] req_id = 4c29747a-5370-4c4d-a7b6-7e0a06e55440
161832025-04-30T06:44:28.341ZDEBGtest_utilization (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgs._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 48759, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }], valid_until: Instant { tv_sec: 4333, tv_nsec: 140067703 } })
161842025-04-30T06:44:28.341ZTRACtest_utilization (DnsResolver): lookup_service_targets: looking up SRV target
161852025-04-30T06:44:28.342ZTRACtest_utilization (dns): buffer buffer = Length: 108 (0x6c) bytes\n0000: 52 90 01 00 00 01 00 00 00 00 00 01 2d 64 65 6e R...........-den\n0010: 64 72 69 74 65 2d 62 36 64 36 35 33 34 31 2d 31 drite-b6d65341-1\n0020: 36 37 63 2d 34 31 64 66 2d 39 62 35 63 2d 34 31 67c-41df-9b5c-41\n0030: 63 64 65 64 39 39 63 32 32 39 04 68 6f 73 74 0d cded99c229.host.\n0040: 63 6f 6e 74 72 6f 6c 2d 70 6c 61 6e 65 05 6f 78 control-plane.ox\n0050: 69 64 65 08 69 6e 74 65 72 6e 61 6c 00 00 1c 00 ide.internal....\n0060: 01 00 00 29 04 d0 00 00 00 00 00 00 ...)........ kind = dns peer_addr = [::1]:62282 req_id = 2cf90e14-2da4-4453-9c30-98bb0a27ca26
161862025-04-30T06:44:28.342ZDEBGtest_utilization (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 21136,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n query_type: AAAA,\n query_class: IN,\n },\n },\n original: [\n 45,\n 100,\n 101,\n 110,\n 100,\n 114,\n 105,\n 116,\n 101,\n 45,\n 98,\n 54,\n 100,\n 54,\n 53,\n 51,\n 52,\n 49,\n 45,\n 49,\n 54,\n 55,\n 99,\n 45,\n 52,\n 49,\n 100,\n 102,\n 45,\n 57,\n 98,\n 53,\n 99,\n 45,\n 52,\n 49,\n 99,\n 100,\n 101,\n 100,\n 57,\n 57,\n 99,\n 50,\n 50,\n 57,\n 4,\n 104,\n 111,\n 115,\n 116,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 28,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:62282 req_id = 2cf90e14-2da4-4453-9c30-98bb0a27ca26
161872025-04-30T06:44:28.342ZDEBGtest_utilization (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
161882025-04-30T06:44:28.342ZDEBGtest_utilization (dns): dns response additional_records = [] kind = dns peer_addr = [::1]:62282 query = LowerQuery { name: LowerName(Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.")), original: Query { name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), query_type: AAAA, query_class: IN } } records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] req_id = 2cf90e14-2da4-4453-9c30-98bb0a27ca26
161892025-04-30T06:44:28.400ZTRACtest_utilization (DnsResolver): lookup_all_socket_v6 srv dns_name = _clickhouse-admin-keeper._tcp.control-plane.oxide.internal
161902025-04-30T06:44:28.401ZTRACtest_utilization (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory
161912025-04-30T06:44:28.401ZTRACtest_utilization (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
161922025-04-30T06:44:28.401ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
161932025-04-30T06:44:28.401ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
161942025-04-30T06:44:28.402ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
161952025-04-30T06:44:28.402ZDEBGtest_utilization (results-sink): inserting 1 samples into database collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
161962025-04-30T06:44:28.403ZDEBGtest_utilization (results-sink-cluster): inserting 1 samples into database collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
161972025-04-30T06:44:28.403ZDEBGtest_utilization (clickhouse-client): unrolling 1 total samples collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 4d1485f0-110e-4161-9a33-c4610d68be5d
161982025-04-30T06:44:28.404ZTRACtest_utilization (clickhouse-client): inserting data collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 4d1485f0-110e-4161-9a33-c4610d68be5d n_columns = 4 n_rows = 2 sql = INSERT INTO oximeter.fields_string FORMAT Native
161992025-04-30T06:44:28.404ZTRACtest_utilization (dns): buffer buffer = Length: 87 (0x57) bytes\n0000: 3d 28 01 00 00 01 00 00 00 00 00 01 18 5f 63 6c =(..........._cl\n0010: 69 63 6b 68 6f 75 73 65 2d 61 64 6d 69 6e 2d 6b ickhouse-admin-k\n0020: 65 65 70 65 72 04 5f 74 63 70 0d 63 6f 6e 74 72 eeper._tcp.contr\n0030: 6f 6c 2d 70 6c 61 6e 65 05 6f 78 69 64 65 08 69 ol-plane.oxide.i\n0040: 6e 74 65 72 6e 61 6c 00 00 21 00 01 00 00 29 04 nternal..!....).\n0050: d0 00 00 00 00 00 00 ....... kind = dns peer_addr = [::1]:64580 req_id = be131a79-50cd-4ae6-a6f2-806f9c87b58e
162002025-04-30T06:44:28.404ZDEBGtest_utilization (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 15656,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 24,\n 95,\n 99,\n 108,\n 105,\n 99,\n 107,\n 104,\n 111,\n 117,\n 115,\n 101,\n 45,\n 97,\n 100,\n 109,\n 105,\n 110,\n 45,\n 107,\n 101,\n 101,\n 112,\n 101,\n 114,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:64580 req_id = be131a79-50cd-4ae6-a6f2-806f9c87b58e
162012025-04-30T06:44:28.405ZDEBGtest_utilization (store): query key key = _clickhouse-admin-keeper._tcp
162022025-04-30T06:44:28.405ZERROtest_utilization (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 15656, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 1, }, query: WireQuery { query: LowerQuery { name: LowerName( Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), ), original: Query { name: Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN, }, }, original: [ 24, 95, 99, 108, 105, 99, 107, 104, 111, 117, 115, 101, 45, 97, 100, 109, 105, 110, 45, 107, 101, 101, 112, 101, 114, 4, 95, 116, 99, 112, 13, 99, 111, 110, 116, 114, 111, 108, 45, 112, 108, 97, 110, 101, 5, 111, 120, 105, 100, 101, 8, 105, 110, 116, 101, 114, 110, 97, 108, 0, 0, 33, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: Some( Edns { rcode_high: 0, version: 0, dnssec_ok: false, max_payload: 1232, options: OPT { options: {}, }, }, ), } NXDOMAIN: no records found for name: "_clickhouse-admin-keeper._tcp.control-plane.oxide.internal." kind = dns peer_addr = [::1]:64580 req_id = be131a79-50cd-4ae6-a6f2-806f9c87b58e
162032025-04-30T06:44:28.406ZDEBGtest_utilization (BackgroundTasks): begin collection background_task = inventory_collection
162042025-04-30T06:44:28.406ZDEBGtest_utilization (BackgroundTasks): begin collection from MGS background_task = inventory_collection mgs_url = http://[::1]:48759
162052025-04-30T06:44:28.406ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/ignition
162062025-04-30T06:44:28.406ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
162072025-04-30T06:44:28.414ZDEBGtest_utilization (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory result = Ok(())
162082025-04-30T06:44:28.414ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
162092025-04-30T06:44:28.414ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
162102025-04-30T06:44:28.414ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
162112025-04-30T06:44:28.419ZDEBGtest_utilization (clickhouse-client): inserted rows into table collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 4d1485f0-110e-4161-9a33-c4610d68be5d n_rows = 2 table_name = fields_string
162122025-04-30T06:44:28.419ZTRACtest_utilization (clickhouse-client): inserting data collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 4d1485f0-110e-4161-9a33-c4610d68be5d n_columns = 4 n_rows = 1 sql = INSERT INTO oximeter.measurements_i64 FORMAT Native
162132025-04-30T06:44:28.420ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
162142025-04-30T06:44:28.420ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
162152025-04-30T06:44:28.421ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
162162025-04-30T06:44:28.423ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/ignition", status: 200, headers: {"content-type": "application/json", "x-request-id": "584641e7-e0ab-4da9-a1dd-75006db55c0c", "content-length": "835", "date": "Wed, 30 Apr 2025 06:44:27 GMT"} })
162172025-04-30T06:44:28.423ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0
162182025-04-30T06:44:28.431ZDEBGtest_utilization (clickhouse-client): inserted rows into table collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 4d1485f0-110e-4161-9a33-c4610d68be5d n_rows = 1 table_name = measurements_i64
162192025-04-30T06:44:28.431ZTRACtest_utilization (results-sink): successfully inserted samples collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
162202025-04-30T06:44:28.432ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "a074cb08-be17-4b81-bce4-2fad96532e13", "content-length": "734", "date": "Wed, 30 Apr 2025 06:44:27 GMT"} })
162212025-04-30T06:44:28.433ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0/component/sp/caboose?firmware_slot=0
162222025-04-30T06:44:28.435ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "774af8a9-744f-4c62-bf1f-86bfa658385e", "content-length": "111", "date": "Wed, 30 Apr 2025 06:44:27 GMT"} })
162232025-04-30T06:44:28.435ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0/component/sp/caboose?firmware_slot=1
162242025-04-30T06:44:28.480ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "1000a72a-1109-4c40-9f34-1d58b3c42592", "content-length": "111", "date": "Wed, 30 Apr 2025 06:44:27 GMT"} })
162252025-04-30T06:44:28.480ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0/component/rot/caboose?firmware_slot=0
162262025-04-30T06:44:28.485ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "328b6352-40de-4782-8e06-b61f7aba3d11", "content-length": "170", "date": "Wed, 30 Apr 2025 06:44:27 GMT"} })
162272025-04-30T06:44:28.485ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0/component/rot/caboose?firmware_slot=1
162282025-04-30T06:44:28.488ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "e8d36b60-d854-4811-ba84-8877c660d61d", "content-length": "170", "date": "Wed, 30 Apr 2025 06:44:27 GMT"} })
162292025-04-30T06:44:28.488ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0/component/stage0/caboose?firmware_slot=0
162302025-04-30T06:44:28.491ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "b62d365d-c9a4-427a-9966-0fc76e2f4455", "content-length": "179", "date": "Wed, 30 Apr 2025 06:44:27 GMT"} })
162312025-04-30T06:44:28.492ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0/component/stage0/caboose?firmware_slot=1
162322025-04-30T06:44:28.505ZWARNtest_utilization (results-sink-cluster): failed to insert some results into metric DB collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 error = Failed to check out connection to database: No backends found for this service file = oximeter/collector/src/results_sink.rs:92
162332025-04-30T06:44:28.505ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "63906aea-3689-4384-b2bc-9a1d46991bef", "content-length": "179", "date": "Wed, 30 Apr 2025 06:44:27 GMT"} })
162342025-04-30T06:44:28.505ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0/component/rot/cmpa
162352025-04-30T06:44:28.508ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "6c52ba57-9307-4e37-81e1-653dec67e782", "content-length": "702", "date": "Wed, 30 Apr 2025 06:44:27 GMT"} })
162362025-04-30T06:44:28.508ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0/component/rot/cfpa
162372025-04-30T06:44:28.508ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "f21e060e-8e1e-4c2b-86ff-6ba8f3b40905", "content-length": "727", "date": "Wed, 30 Apr 2025 06:44:27 GMT"} })
162382025-04-30T06:44:28.508ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0/component/rot/cfpa
162392025-04-30T06:44:28.512ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "b23f17f3-ad28-4ad5-851b-acb390b4b73d", "content-length": "729", "date": "Wed, 30 Apr 2025 06:44:27 GMT"} })
162402025-04-30T06:44:28.512ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0/component/rot/cfpa
162412025-04-30T06:44:28.512ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "a6aa8919-90f6-4ba3-9a2f-44ebe000158e", "content-length": "728", "date": "Wed, 30 Apr 2025 06:44:27 GMT"} })
162422025-04-30T06:44:28.512ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1
162432025-04-30T06:44:28.512ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1", status: 200, headers: {"content-type": "application/json", "x-request-id": "fb6d71af-cba9-4d04-8cff-3147db89faa8", "content-length": "734", "date": "Wed, 30 Apr 2025 06:44:27 GMT"} })
162442025-04-30T06:44:28.512ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1/component/sp/caboose?firmware_slot=0
162452025-04-30T06:44:28.523ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "fe41f22e-276e-4484-a3cc-11867e690c0e", "content-length": "111", "date": "Wed, 30 Apr 2025 06:44:27 GMT"} })
162462025-04-30T06:44:28.523ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1/component/sp/caboose?firmware_slot=1
162472025-04-30T06:44:28.526ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "e0df0d9e-7d21-4b74-b8b4-d9554c6fb60f", "content-length": "111", "date": "Wed, 30 Apr 2025 06:44:27 GMT"} })
162482025-04-30T06:44:28.526ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1/component/rot/caboose?firmware_slot=0
162492025-04-30T06:44:28.527ZDEBGtest_utilization: attempt to fetch latest inventory collection result = Ok("found")
162502025-04-30T06:44:28.529ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "2158000b-060d-4681-b99b-786824182515", "content-length": "170", "date": "Wed, 30 Apr 2025 06:44:27 GMT"} })
162512025-04-30T06:44:28.529ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1/component/rot/caboose?firmware_slot=1
162522025-04-30T06:44:28.541ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "c0b337d2-f11e-445c-8a5a-711bc84d9cf5", "content-length": "170", "date": "Wed, 30 Apr 2025 06:44:27 GMT"} })
162532025-04-30T06:44:28.542ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1/component/stage0/caboose?firmware_slot=0
162542025-04-30T06:44:28.545ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "48f83008-7086-4134-9b63-3201cc2304c7", "content-length": "179", "date": "Wed, 30 Apr 2025 06:44:27 GMT"} })
162552025-04-30T06:44:28.545ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1/component/stage0/caboose?firmware_slot=1
162562025-04-30T06:44:28.547ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "5140f704-94fb-4c97-9587-d1c610a6b98b", "content-length": "179", "date": "Wed, 30 Apr 2025 06:44:27 GMT"} })
162572025-04-30T06:44:28.547ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1/component/rot/cmpa
162582025-04-30T06:44:28.548ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "bccf69c7-a273-4c86-9105-1ab290cab050", "content-length": "702", "date": "Wed, 30 Apr 2025 06:44:27 GMT"} })
162592025-04-30T06:44:28.548ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1/component/rot/cfpa
162602025-04-30T06:44:28.549ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "815b6cb6-c41a-49db-acaf-46fda8713f12", "content-length": "727", "date": "Wed, 30 Apr 2025 06:44:27 GMT"} })
162612025-04-30T06:44:28.550ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1/component/rot/cfpa
162622025-04-30T06:44:28.551ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "1d295660-83ff-4359-bb63-c28df128fc92", "content-length": "729", "date": "Wed, 30 Apr 2025 06:44:27 GMT"} })
162632025-04-30T06:44:28.551ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1/component/rot/cfpa
162642025-04-30T06:44:28.552ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "0b1afaa6-da7b-4b92-bfd4-3ae656022960", "content-length": "728", "date": "Wed, 30 Apr 2025 06:44:27 GMT"} })
162652025-04-30T06:44:28.552ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0
162662025-04-30T06:44:28.561ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "830578f5-f2db-4815-ad9d-6977a5b089f9", "content-length": "723", "date": "Wed, 30 Apr 2025 06:44:27 GMT"} })
162672025-04-30T06:44:28.561ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0/component/sp/caboose?firmware_slot=0
162682025-04-30T06:44:28.577ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "a7aac2c9-6f5b-4751-93d8-5df1a00ea700", "content-length": "109", "date": "Wed, 30 Apr 2025 06:44:27 GMT"} })
162692025-04-30T06:44:28.577ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0/component/sp/caboose?firmware_slot=1
162702025-04-30T06:44:28.577ZTRACtest_utilization (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory
162712025-04-30T06:44:28.577ZTRACtest_utilization (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
162722025-04-30T06:44:28.578ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
162732025-04-30T06:44:28.578ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
162742025-04-30T06:44:28.578ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
162752025-04-30T06:44:28.603ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
162762025-04-30T06:44:28.615ZDEBGtest_utilization (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory result = Ok(())
162772025-04-30T06:44:28.615ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
162782025-04-30T06:44:28.615ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
162792025-04-30T06:44:28.615ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
162802025-04-30T06:44:28.619ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
162812025-04-30T06:44:28.619ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
162822025-04-30T06:44:28.619ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
162832025-04-30T06:44:28.636ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "f88d7b52-5e69-47dc-a385-52afa2201a0a", "content-length": "109", "date": "Wed, 30 Apr 2025 06:44:27 GMT"} })
162842025-04-30T06:44:28.637ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0/component/rot/caboose?firmware_slot=0
162852025-04-30T06:44:28.643ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "d20a14fd-9ad0-4fbe-beb9-5d4ae1b23753", "content-length": "169", "date": "Wed, 30 Apr 2025 06:44:27 GMT"} })
162862025-04-30T06:44:28.644ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0/component/rot/caboose?firmware_slot=1
162872025-04-30T06:44:28.649ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "87fbccda-4a55-46a1-9603-24b449824040", "content-length": "169", "date": "Wed, 30 Apr 2025 06:44:27 GMT"} })
162882025-04-30T06:44:28.650ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0/component/stage0/caboose?firmware_slot=0
162892025-04-30T06:44:28.667ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "c05379d3-d49b-4288-aaf7-2151873410fb", "content-length": "178", "date": "Wed, 30 Apr 2025 06:44:27 GMT"} })
162902025-04-30T06:44:28.667ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0/component/stage0/caboose?firmware_slot=1
162912025-04-30T06:44:28.669ZDEBGtest_utilization: attempt to fetch latest inventory collection result = Ok("found")
162922025-04-30T06:44:28.671ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "6404a428-d120-4ff3-af3a-7bb5cee76802", "content-length": "178", "date": "Wed, 30 Apr 2025 06:44:27 GMT"} })
162932025-04-30T06:44:28.672ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0/component/rot/cmpa
162942025-04-30T06:44:28.674ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "d937db52-4d07-4f93-bb5c-b14bdc756f10", "content-length": "702", "date": "Wed, 30 Apr 2025 06:44:27 GMT"} })
162952025-04-30T06:44:28.674ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0/component/rot/cfpa
162962025-04-30T06:44:28.677ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "f3c4b207-122a-4db2-8563-27171ce13f07", "content-length": "727", "date": "Wed, 30 Apr 2025 06:44:27 GMT"} })
162972025-04-30T06:44:28.678ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0/component/rot/cfpa
162982025-04-30T06:44:28.684ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "e581d3f0-83d0-4041-9045-0f64cf882db1", "content-length": "729", "date": "Wed, 30 Apr 2025 06:44:27 GMT"} })
162992025-04-30T06:44:28.684ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0/component/rot/cfpa
163002025-04-30T06:44:28.686ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "5f28ef4c-8329-4bde-ad48-dfadab721334", "content-length": "728", "date": "Wed, 30 Apr 2025 06:44:27 GMT"} })
163012025-04-30T06:44:28.686ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1
163022025-04-30T06:44:28.688ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1", status: 200, headers: {"content-type": "application/json", "x-request-id": "658131ad-ced1-460d-9cf0-85e1d47943a6", "content-length": "723", "date": "Wed, 30 Apr 2025 06:44:27 GMT"} })
163032025-04-30T06:44:28.689ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1/component/sp/caboose?firmware_slot=0
163042025-04-30T06:44:28.692ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "dea51826-56c2-462c-bb2d-9004bf9496e2", "content-length": "109", "date": "Wed, 30 Apr 2025 06:44:27 GMT"} })
163052025-04-30T06:44:28.692ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1/component/sp/caboose?firmware_slot=1
163062025-04-30T06:44:28.697ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "1effd626-19d4-492c-a79e-2b391242d7cb", "content-length": "109", "date": "Wed, 30 Apr 2025 06:44:27 GMT"} })
163072025-04-30T06:44:28.699ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1/component/rot/caboose?firmware_slot=0
163082025-04-30T06:44:28.702ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "d4567a67-2160-411e-a687-eae0ade35a4d", "content-length": "169", "date": "Wed, 30 Apr 2025 06:44:27 GMT"} })
163092025-04-30T06:44:28.702ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1/component/rot/caboose?firmware_slot=1
163102025-04-30T06:44:28.707ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "ae8c8a67-76f7-479d-bdef-217df7f16eae", "content-length": "169", "date": "Wed, 30 Apr 2025 06:44:27 GMT"} })
163112025-04-30T06:44:28.707ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1/component/stage0/caboose?firmware_slot=0
163122025-04-30T06:44:28.710ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "c3a259f7-c14d-47c7-8a18-57c540d5c420", "content-length": "178", "date": "Wed, 30 Apr 2025 06:44:27 GMT"} })
163132025-04-30T06:44:28.710ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1/component/stage0/caboose?firmware_slot=1
163142025-04-30T06:44:28.712ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "490cec59-cbc8-4068-b48e-0188b39edc66", "content-length": "178", "date": "Wed, 30 Apr 2025 06:44:27 GMT"} })
163152025-04-30T06:44:28.713ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1/component/rot/cmpa
163162025-04-30T06:44:28.715ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "d3157ecb-151d-4b5c-bde4-824635ef87e5", "content-length": "702", "date": "Wed, 30 Apr 2025 06:44:27 GMT"} })
163172025-04-30T06:44:28.716ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1/component/rot/cfpa
163182025-04-30T06:44:28.717ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "c174c77d-36da-41cf-9ed0-4ae2a6d2a6c2", "content-length": "727", "date": "Wed, 30 Apr 2025 06:44:27 GMT"} })
163192025-04-30T06:44:28.718ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1/component/rot/cfpa
163202025-04-30T06:44:28.718ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "db481eba-eae7-48e3-b5bb-75875d401951", "content-length": "729", "date": "Wed, 30 Apr 2025 06:44:27 GMT"} })
163212025-04-30T06:44:28.719ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1/component/rot/cfpa
163222025-04-30T06:44:28.719ZTRACtest_utilization (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory
163232025-04-30T06:44:28.719ZTRACtest_utilization (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
163242025-04-30T06:44:28.720ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
163252025-04-30T06:44:28.720ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
163262025-04-30T06:44:28.721ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
163272025-04-30T06:44:28.721ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "5a9ae48b-3f35-4506-a53f-1e1f3a5cbb53", "content-length": "728", "date": "Wed, 30 Apr 2025 06:44:27 GMT"} })
163282025-04-30T06:44:28.722ZTRACtest_utilization (BackgroundTasks): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet
163292025-04-30T06:44:28.722ZTRACtest_utilization (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
163302025-04-30T06:44:28.722ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
163312025-04-30T06:44:28.722ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
163322025-04-30T06:44:28.723ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
163332025-04-30T06:44:28.725ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
163342025-04-30T06:44:28.733ZDEBGtest_utilization (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory result = Ok(())
163352025-04-30T06:44:28.733ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
163362025-04-30T06:44:28.733ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
163372025-04-30T06:44:28.734ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
163382025-04-30T06:44:28.736ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
163392025-04-30T06:44:28.741ZDEBGtest_utilization (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet result = Ok(())
163402025-04-30T06:44:28.741ZTRACtest_utilization (BackgroundTasks): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet
163412025-04-30T06:44:28.741ZTRACtest_utilization (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
163422025-04-30T06:44:28.741ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
163432025-04-30T06:44:28.741ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
163442025-04-30T06:44:28.742ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
163452025-04-30T06:44:28.745ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
163462025-04-30T06:44:28.745ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
163472025-04-30T06:44:28.745ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
163482025-04-30T06:44:28.746ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
163492025-04-30T06:44:28.750ZDEBGtest_utilization (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet result = Ok(())
163502025-04-30T06:44:28.750ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
163512025-04-30T06:44:28.750ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
163522025-04-30T06:44:28.751ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
163532025-04-30T06:44:28.821ZDEBGtest_utilization (BackgroundTasks): begin collection from Sled Agent background_task = inventory_collection sled_agent_url = http://[::1]:39365
163542025-04-30T06:44:28.821ZDEBGtest_utilization (BackgroundTasks): client request SledAgent = http://[::1]:39365 background_task = inventory_collection body = None method = GET uri = http://[::1]:39365/inventory
163552025-04-30T06:44:28.826ZINFOtest_utilization (dropshot): accepted connection local_addr = [::1]:39365 remote_addr = [::1]:40679 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
163562025-04-30T06:44:28.826ZTRACtest_utilization (dropshot): incoming request local_addr = [::1]:39365 method = GET remote_addr = [::1]:40679 req_id = ba754b03-6bd3-43ec-b8cc-e065fe20dcab sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /inventory
163572025-04-30T06:44:28.827ZINFOtest_utilization (dropshot): request completed latency_us = 430 local_addr = [::1]:39365 method = GET remote_addr = [::1]:40679 req_id = ba754b03-6bd3-43ec-b8cc-e065fe20dcab response_code = 200 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /inventory
163582025-04-30T06:44:28.838ZDEBGtest_utilization (BackgroundTasks): client response SledAgent = http://[::1]:39365 background_task = inventory_collection result = Ok(Response { url: "http://[::1]:39365/inventory", status: 200, headers: {"content-type": "application/json", "x-request-id": "ba754b03-6bd3-43ec-b8cc-e065fe20dcab", "content-length": "3141", "date": "Wed, 30 Apr 2025 06:44:27 GMT"} })
163592025-04-30T06:44:28.845ZDEBGtest_utilization (BackgroundTasks): begin collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
163602025-04-30T06:44:28.845ZDEBGtest_utilization (BackgroundTasks): end collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
163612025-04-30T06:44:28.845ZDEBGtest_utilization (BackgroundTasks): finished collection background_task = inventory_collection
163622025-04-30T06:44:28.847ZTRACtest_utilization (BackgroundTasks): authorize begin action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory
163632025-04-30T06:44:28.847ZTRACtest_utilization (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
163642025-04-30T06:44:28.847ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
163652025-04-30T06:44:28.847ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
163662025-04-30T06:44:28.848ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
163672025-04-30T06:44:28.851ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
163682025-04-30T06:44:28.859ZDEBGtest_utilization (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory result = Ok(())
163692025-04-30T06:44:28.859ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
163702025-04-30T06:44:28.859ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
163712025-04-30T06:44:28.860ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
163722025-04-30T06:44:28.892ZDEBGtest_utilization: attempt to fetch latest inventory collection result = Ok("found")
163732025-04-30T06:44:28.896ZWARNtest_utilization (datastore): Non-retryable transaction failure err = DatabaseError(UniqueViolation, "duplicate key value violates unique constraint \\"inv_caboose_pkey\\"") transaction = inventory_insert_collection
163742025-04-30T06:44:28.896ZWARNtest_utilization (BackgroundTasks): inventory collection failed background_task = inventory_collection error = failed to collect inventory: saving inventory to database: Internal Error: unexpected database error: duplicate key value violates unique constraint "inv_caboose_pkey"
163752025-04-30T06:44:28.896ZDEBGtest_utilization (BackgroundTasks): activation complete background_task = inventory_collection elapsed = 573.285033ms iteration = 47 status = {"error":"failed to collect inventory: saving inventory to database: Internal Error: unexpected database error: duplicate key value violates unique constraint \\"inv_caboose_pkey\\""}
163762025-04-30T06:44:28.896ZDEBGtest_utilization (BackgroundTasks): activating background_task = inventory_collection iteration = 48 reason = Signaled
163772025-04-30T06:44:28.896ZTRACtest_utilization (BackgroundTasks): authorize begin action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory
163782025-04-30T06:44:28.896ZTRACtest_utilization (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
163792025-04-30T06:44:28.896ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
163802025-04-30T06:44:28.897ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
163812025-04-30T06:44:28.902ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
163822025-04-30T06:44:28.906ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
163832025-04-30T06:44:28.918ZDEBGtest_utilization (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory result = Ok(())
163842025-04-30T06:44:28.918ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
163852025-04-30T06:44:28.918ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
163862025-04-30T06:44:28.920ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
163872025-04-30T06:44:28.924ZDEBGtest_utilization (BackgroundTasks): inventory_prune_one: nothing eligible for removal (too few) background_task = inventory_collection candidates = [(13d8b8d9-6c6b-47a6-9190-952544edb75b, 0)]
163882025-04-30T06:44:28.924ZTRACtest_utilization (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal
163892025-04-30T06:44:28.925ZTRACtest_utilization (dns): buffer buffer = Length: 67 (0x43) bytes\n0000: 2a 70 01 00 00 01 00 00 00 00 00 01 04 5f 6d 67 *p..........._mg\n0010: 73 04 5f 74 63 70 0d 63 6f 6e 74 72 6f 6c 2d 70 s._tcp.control-p\n0020: 6c 61 6e 65 05 6f 78 69 64 65 08 69 6e 74 65 72 lane.oxide.inter\n0030: 6e 61 6c 00 00 21 00 01 00 00 29 04 d0 00 00 00 nal..!....).....\n0040: 00 00 00 ... kind = dns peer_addr = [::1]:53194 req_id = dc8e43cd-9c03-40c1-86ff-451e84317438
163902025-04-30T06:44:28.925ZDEBGtest_utilization (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 10864,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_mgs._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_mgs._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 4,\n 95,\n 109,\n 103,\n 115,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:53194 req_id = dc8e43cd-9c03-40c1-86ff-451e84317438
163912025-04-30T06:44:28.925ZDEBGtest_utilization (store): query key key = _mgs._tcp
163922025-04-30T06:44:28.926ZDEBGtest_utilization (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
163932025-04-30T06:44:28.926ZDEBGtest_utilization (dns): dns response additional_records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal"), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] kind = dns peer_addr = [::1]:53194 query = LowerQuery { name: LowerName(Name("_mgs._tcp.control-plane.oxide.internal.")), original: Query { name: Name("_mgs._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN } } records = [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 48759, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal") })) }] req_id = dc8e43cd-9c03-40c1-86ff-451e84317438
163942025-04-30T06:44:28.928ZDEBGtest_utilization (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgs._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 48759, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }], valid_until: Instant { tv_sec: 4333, tv_nsec: 727696435 } })
163952025-04-30T06:44:28.928ZTRACtest_utilization (DnsResolver): lookup_service_targets: looking up SRV target
163962025-04-30T06:44:28.929ZTRACtest_utilization (dns): buffer buffer = Length: 108 (0x6c) bytes\n0000: e2 a0 01 00 00 01 00 00 00 00 00 01 2d 64 65 6e ............-den\n0010: 64 72 69 74 65 2d 62 36 64 36 35 33 34 31 2d 31 drite-b6d65341-1\n0020: 36 37 63 2d 34 31 64 66 2d 39 62 35 63 2d 34 31 67c-41df-9b5c-41\n0030: 63 64 65 64 39 39 63 32 32 39 04 68 6f 73 74 0d cded99c229.host.\n0040: 63 6f 6e 74 72 6f 6c 2d 70 6c 61 6e 65 05 6f 78 control-plane.ox\n0050: 69 64 65 08 69 6e 74 65 72 6e 61 6c 00 00 1c 00 ide.internal....\n0060: 01 00 00 29 04 d0 00 00 00 00 00 00 ...)........ kind = dns peer_addr = [::1]:62996 req_id = b35d3d5f-02e1-4687-a4ad-08e2b696f3b1
163972025-04-30T06:44:28.929ZDEBGtest_utilization (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 58016,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n query_type: AAAA,\n query_class: IN,\n },\n },\n original: [\n 45,\n 100,\n 101,\n 110,\n 100,\n 114,\n 105,\n 116,\n 101,\n 45,\n 98,\n 54,\n 100,\n 54,\n 53,\n 51,\n 52,\n 49,\n 45,\n 49,\n 54,\n 55,\n 99,\n 45,\n 52,\n 49,\n 100,\n 102,\n 45,\n 57,\n 98,\n 53,\n 99,\n 45,\n 52,\n 49,\n 99,\n 100,\n 101,\n 100,\n 57,\n 57,\n 99,\n 50,\n 50,\n 57,\n 4,\n 104,\n 111,\n 115,\n 116,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 28,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:62996 req_id = b35d3d5f-02e1-4687-a4ad-08e2b696f3b1
163982025-04-30T06:44:28.929ZDEBGtest_utilization (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
163992025-04-30T06:44:28.929ZDEBGtest_utilization (dns): dns response additional_records = [] kind = dns peer_addr = [::1]:62996 query = LowerQuery { name: LowerName(Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.")), original: Query { name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), query_type: AAAA, query_class: IN } } records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] req_id = b35d3d5f-02e1-4687-a4ad-08e2b696f3b1
164002025-04-30T06:44:28.982ZTRACtest_utilization (DnsResolver): lookup_all_socket_v6 srv dns_name = _clickhouse-admin-keeper._tcp.control-plane.oxide.internal
164012025-04-30T06:44:28.983ZTRACtest_utilization (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory
164022025-04-30T06:44:28.983ZTRACtest_utilization (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
164032025-04-30T06:44:28.983ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
164042025-04-30T06:44:28.983ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
164052025-04-30T06:44:28.984ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
164062025-04-30T06:44:28.984ZTRACtest_utilization (dns): buffer buffer = Length: 87 (0x57) bytes\n0000: b1 49 01 00 00 01 00 00 00 00 00 01 18 5f 63 6c .I..........._cl\n0010: 69 63 6b 68 6f 75 73 65 2d 61 64 6d 69 6e 2d 6b ickhouse-admin-k\n0020: 65 65 70 65 72 04 5f 74 63 70 0d 63 6f 6e 74 72 eeper._tcp.contr\n0030: 6f 6c 2d 70 6c 61 6e 65 05 6f 78 69 64 65 08 69 ol-plane.oxide.i\n0040: 6e 74 65 72 6e 61 6c 00 00 21 00 01 00 00 29 04 nternal..!....).\n0050: d0 00 00 00 00 00 00 ....... kind = dns peer_addr = [::1]:62128 req_id = 467c5d2f-aa72-49b8-91e4-e80f73b92d04
164072025-04-30T06:44:28.984ZDEBGtest_utilization (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 45385,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 24,\n 95,\n 99,\n 108,\n 105,\n 99,\n 107,\n 104,\n 111,\n 117,\n 115,\n 101,\n 45,\n 97,\n 100,\n 109,\n 105,\n 110,\n 45,\n 107,\n 101,\n 101,\n 112,\n 101,\n 114,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:62128 req_id = 467c5d2f-aa72-49b8-91e4-e80f73b92d04
164082025-04-30T06:44:28.984ZDEBGtest_utilization (store): query key key = _clickhouse-admin-keeper._tcp
164092025-04-30T06:44:28.984ZERROtest_utilization (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 45385, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 1, }, query: WireQuery { query: LowerQuery { name: LowerName( Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), ), original: Query { name: Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN, }, }, original: [ 24, 95, 99, 108, 105, 99, 107, 104, 111, 117, 115, 101, 45, 97, 100, 109, 105, 110, 45, 107, 101, 101, 112, 101, 114, 4, 95, 116, 99, 112, 13, 99, 111, 110, 116, 114, 111, 108, 45, 112, 108, 97, 110, 101, 5, 111, 120, 105, 100, 101, 8, 105, 110, 116, 101, 114, 110, 97, 108, 0, 0, 33, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: Some( Edns { rcode_high: 0, version: 0, dnssec_ok: false, max_payload: 1232, options: OPT { options: {}, }, }, ), } NXDOMAIN: no records found for name: "_clickhouse-admin-keeper._tcp.control-plane.oxide.internal." kind = dns peer_addr = [::1]:62128 req_id = 467c5d2f-aa72-49b8-91e4-e80f73b92d04
164102025-04-30T06:44:28.985ZDEBGtest_utilization (BackgroundTasks): begin collection background_task = inventory_collection
164112025-04-30T06:44:28.985ZDEBGtest_utilization (BackgroundTasks): begin collection from MGS background_task = inventory_collection mgs_url = http://[::1]:48759
164122025-04-30T06:44:28.985ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/ignition
164132025-04-30T06:44:28.991ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
164142025-04-30T06:44:28.998ZDEBGtest_utilization (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory result = Ok(())
164152025-04-30T06:44:28.998ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
164162025-04-30T06:44:28.998ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
164172025-04-30T06:44:28.999ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
164182025-04-30T06:44:29.001ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/ignition", status: 200, headers: {"content-type": "application/json", "x-request-id": "5ba37acf-e76e-4b77-9b59-877685a99466", "content-length": "835", "date": "Wed, 30 Apr 2025 06:44:28 GMT"} })
164192025-04-30T06:44:29.001ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0
164202025-04-30T06:44:29.002ZDEBGtest_utilization (collection-task): sent timer-based collection request to the collection task collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
164212025-04-30T06:44:29.002ZDEBGtest_utilization (collection-task): collecting from producer collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
164222025-04-30T06:44:29.005ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
164232025-04-30T06:44:29.005ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
164242025-04-30T06:44:29.006ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
164252025-04-30T06:44:29.006ZTRACtest_utilization (collection-task): sent collection request to producer collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
164262025-04-30T06:44:29.007ZDEBGtest_utilization (collection-task): collected results from producer collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 n_results = 1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
164272025-04-30T06:44:29.007ZTRACtest_utilization (collection-task): forwarded results to main collection loop collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
164282025-04-30T06:44:29.007ZTRACtest_utilization (collection-task): top of inner collection loop, waiting for next request collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
164292025-04-30T06:44:29.010ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "87a60906-0600-400a-9ad1-052dee295aed", "content-length": "734", "date": "Wed, 30 Apr 2025 06:44:28 GMT"} })
164302025-04-30T06:44:29.011ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0/component/sp/caboose?firmware_slot=0
164312025-04-30T06:44:29.026ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "67474938-eb35-432b-88a8-b26c36901251", "content-length": "111", "date": "Wed, 30 Apr 2025 06:44:28 GMT"} })
164322025-04-30T06:44:29.026ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0/component/sp/caboose?firmware_slot=1
164332025-04-30T06:44:29.030ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "cc6e159e-b6a7-4ba5-b6a0-3aa890deb1e1", "content-length": "111", "date": "Wed, 30 Apr 2025 06:44:28 GMT"} })
164342025-04-30T06:44:29.030ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0/component/rot/caboose?firmware_slot=0
164352025-04-30T06:44:29.033ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "644553c7-2a95-4e7a-9f69-d5f587c8abf3", "content-length": "170", "date": "Wed, 30 Apr 2025 06:44:28 GMT"} })
164362025-04-30T06:44:29.033ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0/component/rot/caboose?firmware_slot=1
164372025-04-30T06:44:29.035ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "102797e6-8430-46f7-9ec5-239888a707d2", "content-length": "170", "date": "Wed, 30 Apr 2025 06:44:28 GMT"} })
164382025-04-30T06:44:29.035ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0/component/stage0/caboose?firmware_slot=0
164392025-04-30T06:44:29.041ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "6ac48214-8cf1-46a2-9430-8014f23b1c33", "content-length": "179", "date": "Wed, 30 Apr 2025 06:44:28 GMT"} })
164402025-04-30T06:44:29.041ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0/component/stage0/caboose?firmware_slot=1
164412025-04-30T06:44:29.044ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "e858f509-df52-43bc-bf3a-a1ffc0e90f73", "content-length": "179", "date": "Wed, 30 Apr 2025 06:44:28 GMT"} })
164422025-04-30T06:44:29.044ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0/component/rot/cmpa
164432025-04-30T06:44:29.045ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "83cc7b4c-66f3-4ca9-aae3-88b956fecef5", "content-length": "702", "date": "Wed, 30 Apr 2025 06:44:28 GMT"} })
164442025-04-30T06:44:29.046ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0/component/rot/cfpa
164452025-04-30T06:44:29.047ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "0804b80c-ee2c-4cd1-9d34-77b46780f06c", "content-length": "727", "date": "Wed, 30 Apr 2025 06:44:28 GMT"} })
164462025-04-30T06:44:29.047ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0/component/rot/cfpa
164472025-04-30T06:44:29.055ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "97b7af3e-c673-4c7b-8c7d-b773c0e3d965", "content-length": "729", "date": "Wed, 30 Apr 2025 06:44:28 GMT"} })
164482025-04-30T06:44:29.056ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0/component/rot/cfpa
164492025-04-30T06:44:29.059ZDEBGtest_utilization: attempt to fetch latest inventory collection result = Ok("found")
164502025-04-30T06:44:29.059ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "709f833e-0a73-4c08-9c5e-3da1aaf64f95", "content-length": "728", "date": "Wed, 30 Apr 2025 06:44:28 GMT"} })
164512025-04-30T06:44:29.059ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1
164522025-04-30T06:44:29.064ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1", status: 200, headers: {"content-type": "application/json", "x-request-id": "af343320-d8aa-407b-8d13-0a8180bf715d", "content-length": "734", "date": "Wed, 30 Apr 2025 06:44:28 GMT"} })
164532025-04-30T06:44:29.065ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1/component/sp/caboose?firmware_slot=0
164542025-04-30T06:44:29.069ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "81171465-64f2-401f-bb9f-82dd7343edab", "content-length": "111", "date": "Wed, 30 Apr 2025 06:44:28 GMT"} })
164552025-04-30T06:44:29.071ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1/component/sp/caboose?firmware_slot=1
164562025-04-30T06:44:29.073ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "15951ad0-7404-4fad-a1d6-670f3787883a", "content-length": "111", "date": "Wed, 30 Apr 2025 06:44:28 GMT"} })
164572025-04-30T06:44:29.073ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1/component/rot/caboose?firmware_slot=0
164582025-04-30T06:44:29.075ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "1d2b186a-45f7-4246-b16a-c712f2c7df42", "content-length": "170", "date": "Wed, 30 Apr 2025 06:44:28 GMT"} })
164592025-04-30T06:44:29.075ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1/component/rot/caboose?firmware_slot=1
164602025-04-30T06:44:29.079ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "636e5536-e23f-430d-87d2-ebfab4fd2ed0", "content-length": "170", "date": "Wed, 30 Apr 2025 06:44:28 GMT"} })
164612025-04-30T06:44:29.079ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1/component/stage0/caboose?firmware_slot=0
164622025-04-30T06:44:29.082ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "98ef1134-c5b5-4f4f-9b20-7794bc421849", "content-length": "179", "date": "Wed, 30 Apr 2025 06:44:28 GMT"} })
164632025-04-30T06:44:29.082ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1/component/stage0/caboose?firmware_slot=1
164642025-04-30T06:44:29.086ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "5c65bb0b-4d2a-4f9a-b6fd-47bafc23272b", "content-length": "179", "date": "Wed, 30 Apr 2025 06:44:28 GMT"} })
164652025-04-30T06:44:29.087ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1/component/rot/cmpa
164662025-04-30T06:44:29.092ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "57e22157-32cc-4a6b-8648-b4465014c4d0", "content-length": "702", "date": "Wed, 30 Apr 2025 06:44:28 GMT"} })
164672025-04-30T06:44:29.095ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1/component/rot/cfpa
164682025-04-30T06:44:29.096ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "e92e44b5-c2eb-45f2-b4d1-a836002540a9", "content-length": "727", "date": "Wed, 30 Apr 2025 06:44:28 GMT"} })
164692025-04-30T06:44:29.097ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1/component/rot/cfpa
164702025-04-30T06:44:29.101ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "a1c1b2d9-8383-4d10-9282-749c89a91a72", "content-length": "729", "date": "Wed, 30 Apr 2025 06:44:28 GMT"} })
164712025-04-30T06:44:29.102ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1/component/rot/cfpa
164722025-04-30T06:44:29.105ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "0b86694e-4b11-4a1a-98e7-100c0ecc00c6", "content-length": "728", "date": "Wed, 30 Apr 2025 06:44:28 GMT"} })
164732025-04-30T06:44:29.106ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0
164742025-04-30T06:44:29.109ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "5fd0fa0c-96c3-4116-a0cb-31bbc29745ad", "content-length": "723", "date": "Wed, 30 Apr 2025 06:44:28 GMT"} })
164752025-04-30T06:44:29.109ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0/component/sp/caboose?firmware_slot=0
164762025-04-30T06:44:29.110ZTRACtest_utilization (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory
164772025-04-30T06:44:29.110ZTRACtest_utilization (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
164782025-04-30T06:44:29.110ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
164792025-04-30T06:44:29.111ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
164802025-04-30T06:44:29.112ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
164812025-04-30T06:44:29.115ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "fec9d08c-bb35-427e-933e-3368d82b392b", "content-length": "109", "date": "Wed, 30 Apr 2025 06:44:28 GMT"} })
164822025-04-30T06:44:29.115ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0/component/sp/caboose?firmware_slot=1
164832025-04-30T06:44:29.115ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
164842025-04-30T06:44:29.121ZDEBGtest_utilization (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory result = Ok(())
164852025-04-30T06:44:29.121ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
164862025-04-30T06:44:29.121ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
164872025-04-30T06:44:29.121ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
164882025-04-30T06:44:29.124ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
164892025-04-30T06:44:29.124ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
164902025-04-30T06:44:29.130ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
164912025-04-30T06:44:29.132ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "3f85133c-9846-4d5d-b516-6481d591990f", "content-length": "109", "date": "Wed, 30 Apr 2025 06:44:28 GMT"} })
164922025-04-30T06:44:29.132ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0/component/rot/caboose?firmware_slot=0
164932025-04-30T06:44:29.135ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "bb69eb78-6086-4106-ab0e-2fd358e61927", "content-length": "169", "date": "Wed, 30 Apr 2025 06:44:28 GMT"} })
164942025-04-30T06:44:29.136ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0/component/rot/caboose?firmware_slot=1
164952025-04-30T06:44:29.142ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "010a7859-3d53-444d-8065-c424a2d078b8", "content-length": "169", "date": "Wed, 30 Apr 2025 06:44:28 GMT"} })
164962025-04-30T06:44:29.142ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0/component/stage0/caboose?firmware_slot=0
164972025-04-30T06:44:29.153ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "34ddb1c1-a037-4d20-bb5d-8cfb52f421aa", "content-length": "178", "date": "Wed, 30 Apr 2025 06:44:28 GMT"} })
164982025-04-30T06:44:29.153ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0/component/stage0/caboose?firmware_slot=1
164992025-04-30T06:44:29.163ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "227db71d-05ad-453f-9d6b-d2153ab93873", "content-length": "178", "date": "Wed, 30 Apr 2025 06:44:28 GMT"} })
165002025-04-30T06:44:29.163ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0/component/rot/cmpa
165012025-04-30T06:44:29.168ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "ebcdc217-4d03-4850-818b-0ed40d5b1ec7", "content-length": "702", "date": "Wed, 30 Apr 2025 06:44:28 GMT"} })
165022025-04-30T06:44:29.169ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0/component/rot/cfpa
165032025-04-30T06:44:29.171ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "fa76da67-8373-4d18-a793-877e9ade15ac", "content-length": "727", "date": "Wed, 30 Apr 2025 06:44:28 GMT"} })
165042025-04-30T06:44:29.171ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0/component/rot/cfpa
165052025-04-30T06:44:29.175ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "dfb54a8c-826d-4fca-be1c-bfae8ca8ec6f", "content-length": "729", "date": "Wed, 30 Apr 2025 06:44:28 GMT"} })
165062025-04-30T06:44:29.175ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0/component/rot/cfpa
165072025-04-30T06:44:29.176ZDEBGtest_utilization: attempt to fetch latest inventory collection result = Ok("found")
165082025-04-30T06:44:29.189ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "81024cb0-e3b8-460a-a647-1a8b91a7d430", "content-length": "728", "date": "Wed, 30 Apr 2025 06:44:28 GMT"} })
165092025-04-30T06:44:29.189ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1
165102025-04-30T06:44:29.191ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1", status: 200, headers: {"content-type": "application/json", "x-request-id": "365821c5-e23b-4b42-9af2-e188f10f2f70", "content-length": "723", "date": "Wed, 30 Apr 2025 06:44:28 GMT"} })
165112025-04-30T06:44:29.192ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1/component/sp/caboose?firmware_slot=0
165122025-04-30T06:44:29.195ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "0008a76b-5f00-434d-ab7b-efe9f65668fb", "content-length": "109", "date": "Wed, 30 Apr 2025 06:44:28 GMT"} })
165132025-04-30T06:44:29.198ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1/component/sp/caboose?firmware_slot=1
165142025-04-30T06:44:29.200ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "ee8daf6f-3463-4632-b1ed-044909e6bd8d", "content-length": "109", "date": "Wed, 30 Apr 2025 06:44:28 GMT"} })
165152025-04-30T06:44:29.200ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1/component/rot/caboose?firmware_slot=0
165162025-04-30T06:44:29.201ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "b712219d-9bae-4b9e-b7e2-b80294bbbf2a", "content-length": "169", "date": "Wed, 30 Apr 2025 06:44:28 GMT"} })
165172025-04-30T06:44:29.201ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1/component/rot/caboose?firmware_slot=1
165182025-04-30T06:44:29.203ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "798bd04c-da33-4d12-b302-9cf86c13712c", "content-length": "169", "date": "Wed, 30 Apr 2025 06:44:28 GMT"} })
165192025-04-30T06:44:29.203ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1/component/stage0/caboose?firmware_slot=0
165202025-04-30T06:44:29.206ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "0ef216bd-5112-4f48-89a6-4c35a8baec2b", "content-length": "178", "date": "Wed, 30 Apr 2025 06:44:28 GMT"} })
165212025-04-30T06:44:29.207ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1/component/stage0/caboose?firmware_slot=1
165222025-04-30T06:44:29.209ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "b7ce4019-e4b8-4d93-b86d-74ffc569f203", "content-length": "178", "date": "Wed, 30 Apr 2025 06:44:28 GMT"} })
165232025-04-30T06:44:29.209ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1/component/rot/cmpa
165242025-04-30T06:44:29.211ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "4a9b42b1-1d9f-4e52-bda0-8e1abc6b60f6", "content-length": "702", "date": "Wed, 30 Apr 2025 06:44:28 GMT"} })
165252025-04-30T06:44:29.211ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1/component/rot/cfpa
165262025-04-30T06:44:29.212ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "06026b22-2c7a-4a8f-83db-083687d0557e", "content-length": "727", "date": "Wed, 30 Apr 2025 06:44:28 GMT"} })
165272025-04-30T06:44:29.212ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1/component/rot/cfpa
165282025-04-30T06:44:29.213ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "9447004f-9c9d-4399-9812-a675fa25b45b", "content-length": "729", "date": "Wed, 30 Apr 2025 06:44:28 GMT"} })
165292025-04-30T06:44:29.214ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1/component/rot/cfpa
165302025-04-30T06:44:29.215ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "d849aa67-6f69-4b43-81e7-39a1c9563b1c", "content-length": "728", "date": "Wed, 30 Apr 2025 06:44:28 GMT"} })
165312025-04-30T06:44:29.215ZTRACtest_utilization (BackgroundTasks): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet
165322025-04-30T06:44:29.215ZTRACtest_utilization (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
165332025-04-30T06:44:29.215ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
165342025-04-30T06:44:29.215ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
165352025-04-30T06:44:29.217ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
165362025-04-30T06:44:29.229ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
165372025-04-30T06:44:29.241ZDEBGtest_utilization (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet result = Ok(())
165382025-04-30T06:44:29.241ZTRACtest_utilization (BackgroundTasks): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet
165392025-04-30T06:44:29.241ZTRACtest_utilization (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
165402025-04-30T06:44:29.242ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
165412025-04-30T06:44:29.242ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
165422025-04-30T06:44:29.243ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
165432025-04-30T06:44:29.244ZTRACtest_utilization (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory
165442025-04-30T06:44:29.244ZTRACtest_utilization (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
165452025-04-30T06:44:29.244ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
165462025-04-30T06:44:29.244ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
165472025-04-30T06:44:29.245ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
165482025-04-30T06:44:29.247ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
165492025-04-30T06:44:29.252ZDEBGtest_utilization (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet result = Ok(())
165502025-04-30T06:44:29.252ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
165512025-04-30T06:44:29.252ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
165522025-04-30T06:44:29.253ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
165532025-04-30T06:44:29.254ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
165542025-04-30T06:44:29.262ZDEBGtest_utilization (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory result = Ok(())
165552025-04-30T06:44:29.262ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
165562025-04-30T06:44:29.262ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
165572025-04-30T06:44:29.263ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
165582025-04-30T06:44:29.266ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
165592025-04-30T06:44:29.267ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
165602025-04-30T06:44:29.279ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
165612025-04-30T06:44:29.356ZDEBGtest_utilization (BackgroundTasks): begin collection from Sled Agent background_task = inventory_collection sled_agent_url = http://[::1]:39365
165622025-04-30T06:44:29.356ZDEBGtest_utilization (BackgroundTasks): client request SledAgent = http://[::1]:39365 background_task = inventory_collection body = None method = GET uri = http://[::1]:39365/inventory
165632025-04-30T06:44:29.358ZINFOtest_utilization (dropshot): accepted connection local_addr = [::1]:39365 remote_addr = [::1]:42165 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
165642025-04-30T06:44:29.358ZDEBGtest_utilization (results-sink): inserting 1 samples into database collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
165652025-04-30T06:44:29.358ZDEBGtest_utilization (results-sink-cluster): inserting 1 samples into database collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
165662025-04-30T06:44:29.359ZDEBGtest_utilization (clickhouse-client): unrolling 1 total samples collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 4d1485f0-110e-4161-9a33-c4610d68be5d
165672025-04-30T06:44:29.361ZTRACtest_utilization (clickhouse-client): inserting data collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 4d1485f0-110e-4161-9a33-c4610d68be5d n_columns = 4 n_rows = 2 sql = INSERT INTO oximeter.fields_string FORMAT Native
165682025-04-30T06:44:29.362ZTRACtest_utilization (dropshot): incoming request local_addr = [::1]:39365 method = GET remote_addr = [::1]:42165 req_id = 6e97db08-ebd6-465b-882c-25edeb876bb5 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /inventory
165692025-04-30T06:44:29.362ZINFOtest_utilization (dropshot): request completed latency_us = 635 local_addr = [::1]:39365 method = GET remote_addr = [::1]:42165 req_id = 6e97db08-ebd6-465b-882c-25edeb876bb5 response_code = 200 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /inventory
165702025-04-30T06:44:29.363ZDEBGtest_utilization (BackgroundTasks): client response SledAgent = http://[::1]:39365 background_task = inventory_collection result = Ok(Response { url: "http://[::1]:39365/inventory", status: 200, headers: {"content-type": "application/json", "x-request-id": "6e97db08-ebd6-465b-882c-25edeb876bb5", "content-length": "3141", "date": "Wed, 30 Apr 2025 06:44:28 GMT"} })
165712025-04-30T06:44:29.366ZDEBGtest_utilization (BackgroundTasks): begin collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
165722025-04-30T06:44:29.366ZDEBGtest_utilization (BackgroundTasks): end collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
165732025-04-30T06:44:29.366ZDEBGtest_utilization (BackgroundTasks): finished collection background_task = inventory_collection
165742025-04-30T06:44:29.369ZTRACtest_utilization (BackgroundTasks): authorize begin action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory
165752025-04-30T06:44:29.369ZTRACtest_utilization (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
165762025-04-30T06:44:29.369ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
165772025-04-30T06:44:29.369ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
165782025-04-30T06:44:29.370ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
165792025-04-30T06:44:29.373ZDEBGtest_utilization (clickhouse-client): inserted rows into table collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 4d1485f0-110e-4161-9a33-c4610d68be5d n_rows = 2 table_name = fields_string
165802025-04-30T06:44:29.373ZTRACtest_utilization (clickhouse-client): inserting data collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 4d1485f0-110e-4161-9a33-c4610d68be5d n_columns = 4 n_rows = 1 sql = INSERT INTO oximeter.measurements_i64 FORMAT Native
165812025-04-30T06:44:29.378ZDEBGtest_utilization (clickhouse-client): inserted rows into table collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 4d1485f0-110e-4161-9a33-c4610d68be5d n_rows = 1 table_name = measurements_i64
165822025-04-30T06:44:29.379ZTRACtest_utilization (results-sink): successfully inserted samples collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
165832025-04-30T06:44:29.379ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
165842025-04-30T06:44:29.384ZDEBGtest_utilization (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory result = Ok(())
165852025-04-30T06:44:29.384ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
165862025-04-30T06:44:29.384ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
165872025-04-30T06:44:29.385ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
165882025-04-30T06:44:29.413ZDEBGtest_utilization: attempt to fetch latest inventory collection result = Ok("found")
165892025-04-30T06:44:29.473ZTRACtest_utilization (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory
165902025-04-30T06:44:29.473ZTRACtest_utilization (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
165912025-04-30T06:44:29.473ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
165922025-04-30T06:44:29.473ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
165932025-04-30T06:44:29.474ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
165942025-04-30T06:44:29.474ZTRACtest_utilization (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal
165952025-04-30T06:44:29.474ZWARNtest_utilization (results-sink-cluster): failed to insert some results into metric DB collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 error = Failed to check out connection to database: No backends found for this service file = oximeter/collector/src/results_sink.rs:92
165962025-04-30T06:44:29.475ZWARNtest_utilization (datastore): Non-retryable transaction failure err = DatabaseError(UniqueViolation, "duplicate key value violates unique constraint \\"inv_caboose_pkey\\"") transaction = inventory_insert_collection
165972025-04-30T06:44:29.476ZWARNtest_utilization (BackgroundTasks): inventory collection failed background_task = inventory_collection error = failed to collect inventory: saving inventory to database: Internal Error: unexpected database error: duplicate key value violates unique constraint "inv_caboose_pkey"
165982025-04-30T06:44:29.476ZDEBGtest_utilization (BackgroundTasks): activation complete background_task = inventory_collection elapsed = 579.700035ms iteration = 48 status = {"error":"failed to collect inventory: saving inventory to database: Internal Error: unexpected database error: duplicate key value violates unique constraint \\"inv_caboose_pkey\\""}
165992025-04-30T06:44:29.476ZDEBGtest_utilization (BackgroundTasks): activating background_task = inventory_collection iteration = 49 reason = Signaled
166002025-04-30T06:44:29.476ZTRACtest_utilization (BackgroundTasks): authorize begin action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory
166012025-04-30T06:44:29.476ZTRACtest_utilization (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
166022025-04-30T06:44:29.476ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
166032025-04-30T06:44:29.476ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
166042025-04-30T06:44:29.477ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
166052025-04-30T06:44:29.480ZTRACtest_utilization (dns): buffer buffer = Length: 72 (0x48) bytes\n0000: bb 73 01 00 00 01 00 00 00 00 00 01 09 5f 64 65 .s..........._de\n0010: 6e 64 72 69 74 65 04 5f 74 63 70 0d 63 6f 6e 74 ndrite._tcp.cont\n0020: 72 6f 6c 2d 70 6c 61 6e 65 05 6f 78 69 64 65 08 rol-plane.oxide.\n0030: 69 6e 74 65 72 6e 61 6c 00 00 21 00 01 00 00 29 internal..!....)\n0040: 04 d0 00 00 00 00 00 00 ........ kind = dns peer_addr = [::1]:50569 req_id = 3aef3746-82cb-43fd-9a7c-a5a701e4df05
166062025-04-30T06:44:29.480ZDEBGtest_utilization (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 47987,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_dendrite._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_dendrite._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 9,\n 95,\n 100,\n 101,\n 110,\n 100,\n 114,\n 105,\n 116,\n 101,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:50569 req_id = 3aef3746-82cb-43fd-9a7c-a5a701e4df05
166072025-04-30T06:44:29.480ZDEBGtest_utilization (store): query key key = _dendrite._tcp
166082025-04-30T06:44:29.480ZDEBGtest_utilization (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
166092025-04-30T06:44:29.480ZDEBGtest_utilization (dns): dns response additional_records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal"), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] kind = dns peer_addr = [::1]:50569 query = LowerQuery { name: LowerName(Name("_dendrite._tcp.control-plane.oxide.internal.")), original: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN } } records = [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 47974, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal") })) }] req_id = 3aef3746-82cb-43fd-9a7c-a5a701e4df05
166102025-04-30T06:44:29.481ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
166112025-04-30T06:44:29.486ZDEBGtest_utilization (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory result = Ok(())
166122025-04-30T06:44:29.486ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
166132025-04-30T06:44:29.486ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
166142025-04-30T06:44:29.487ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
166152025-04-30T06:44:29.487ZDEBGtest_utilization (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 47974, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }], valid_until: Instant { tv_sec: 4334, tv_nsec: 286175718 } })
166162025-04-30T06:44:29.487ZTRACtest_utilization (DnsResolver): lookup_service_targets: looking up SRV target
166172025-04-30T06:44:29.496ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
166182025-04-30T06:44:29.504ZDEBGtest_utilization (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory result = Ok(())
166192025-04-30T06:44:29.504ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
166202025-04-30T06:44:29.504ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
166212025-04-30T06:44:29.504ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
166222025-04-30T06:44:29.509ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
166232025-04-30T06:44:29.509ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
166242025-04-30T06:44:29.509ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
166252025-04-30T06:44:29.510ZDEBGtest_utilization (BackgroundTasks): inventory_prune_one: nothing eligible for removal (too few) background_task = inventory_collection candidates = [(13d8b8d9-6c6b-47a6-9190-952544edb75b, 0)]
166262025-04-30T06:44:29.510ZTRACtest_utilization (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal
166272025-04-30T06:44:29.514ZTRACtest_utilization (dns): buffer buffer = Length: 108 (0x6c) bytes\n0000: e9 ad 01 00 00 01 00 00 00 00 00 01 2d 64 65 6e ............-den\n0010: 64 72 69 74 65 2d 62 36 64 36 35 33 34 31 2d 31 drite-b6d65341-1\n0020: 36 37 63 2d 34 31 64 66 2d 39 62 35 63 2d 34 31 67c-41df-9b5c-41\n0030: 63 64 65 64 39 39 63 32 32 39 04 68 6f 73 74 0d cded99c229.host.\n0040: 63 6f 6e 74 72 6f 6c 2d 70 6c 61 6e 65 05 6f 78 control-plane.ox\n0050: 69 64 65 08 69 6e 74 65 72 6e 61 6c 00 00 1c 00 ide.internal....\n0060: 01 00 00 29 04 d0 00 00 00 00 00 00 ...)........ kind = dns peer_addr = [::1]:56480 req_id = 32b34adf-7ad2-49af-9151-146441d01ae2
166282025-04-30T06:44:29.516ZDEBGtest_utilization (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 59821,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n query_type: AAAA,\n query_class: IN,\n },\n },\n original: [\n 45,\n 100,\n 101,\n 110,\n 100,\n 114,\n 105,\n 116,\n 101,\n 45,\n 98,\n 54,\n 100,\n 54,\n 53,\n 51,\n 52,\n 49,\n 45,\n 49,\n 54,\n 55,\n 99,\n 45,\n 52,\n 49,\n 100,\n 102,\n 45,\n 57,\n 98,\n 53,\n 99,\n 45,\n 52,\n 49,\n 99,\n 100,\n 101,\n 100,\n 57,\n 57,\n 99,\n 50,\n 50,\n 57,\n 4,\n 104,\n 111,\n 115,\n 116,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 28,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:56480 req_id = 32b34adf-7ad2-49af-9151-146441d01ae2
166292025-04-30T06:44:29.516ZDEBGtest_utilization (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
166302025-04-30T06:44:29.516ZDEBGtest_utilization (dns): dns response additional_records = [] kind = dns peer_addr = [::1]:56480 query = LowerQuery { name: LowerName(Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.")), original: Query { name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), query_type: AAAA, query_class: IN } } records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] req_id = 32b34adf-7ad2-49af-9151-146441d01ae2
166312025-04-30T06:44:29.520ZINFOtest_utilization (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_v4_garbage_collector
166322025-04-30T06:44:29.601ZINFOtest_utilization (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_v4_garbage_collector zone_address = ::1
166332025-04-30T06:44:29.601ZDEBGtest_utilization (MgsClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[::1]:12225/local/switch-id
166342025-04-30T06:44:29.602ZDEBGtest_utilization (MgsClient): client response background_task = nat_v4_garbage_collector result = Err(reqwest::Error { kind: Request, url: "http://[::1]:12225/local/switch-id", source: hyper_util::client::legacy::Error(Connect, ConnectError("tcp connect error", Os { code: 146, kind: ConnectionRefused, message: "Connection refused" })) })
166352025-04-30T06:44:29.602ZWARNtest_utilization (BackgroundTasks): failed to identify switch slot for dendrite background_task = nat_v4_garbage_collector reason = Communication Error: error sending request for url (http://[::1]:12225/local/switch-id): client error (Connect): tcp connect error: Connection refused (os error 146): Connection refused (os error 146) zone_address = ::1
166362025-04-30T06:44:29.606ZWARNtest_utilization (BackgroundTasks): Failed to map switch zone addr: Communication Error: error sending request for url (http://[::1]:12225/local/switch-id), retrying background_task = nat_v4_garbage_collector
166372025-04-30T06:44:29.606ZTRACtest_utilization (dns): buffer buffer = Length: 67 (0x43) bytes\n0000: 99 ac 01 00 00 01 00 00 00 00 00 01 04 5f 6d 67 ............._mg\n0010: 73 04 5f 74 63 70 0d 63 6f 6e 74 72 6f 6c 2d 70 s._tcp.control-p\n0020: 6c 61 6e 65 05 6f 78 69 64 65 08 69 6e 74 65 72 lane.oxide.inter\n0030: 6e 61 6c 00 00 21 00 01 00 00 29 04 d0 00 00 00 nal..!....).....\n0040: 00 00 00 ... kind = dns peer_addr = [::1]:56968 req_id = b2f70367-521a-4cc7-9894-b878637ef5b2
166382025-04-30T06:44:29.606ZDEBGtest_utilization (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 39340,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_mgs._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_mgs._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 4,\n 95,\n 109,\n 103,\n 115,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:56968 req_id = b2f70367-521a-4cc7-9894-b878637ef5b2
166392025-04-30T06:44:29.607ZDEBGtest_utilization (store): query key key = _mgs._tcp
166402025-04-30T06:44:29.607ZDEBGtest_utilization (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
166412025-04-30T06:44:29.607ZDEBGtest_utilization (dns): dns response additional_records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal"), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] kind = dns peer_addr = [::1]:56968 query = LowerQuery { name: LowerName(Name("_mgs._tcp.control-plane.oxide.internal.")), original: Query { name: Name("_mgs._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN } } records = [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 48759, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal") })) }] req_id = b2f70367-521a-4cc7-9894-b878637ef5b2
166422025-04-30T06:44:29.609ZDEBGtest_utilization (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgs._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 48759, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }], valid_until: Instant { tv_sec: 4334, tv_nsec: 407002829 } })
166432025-04-30T06:44:29.609ZTRACtest_utilization (DnsResolver): lookup_service_targets: looking up SRV target
166442025-04-30T06:44:29.609ZTRACtest_utilization (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal
166452025-04-30T06:44:29.611ZTRACtest_utilization (dns): buffer buffer = Length: 108 (0x6c) bytes\n0000: f1 9d 01 00 00 01 00 00 00 00 00 01 2d 64 65 6e ............-den\n0010: 64 72 69 74 65 2d 62 36 64 36 35 33 34 31 2d 31 drite-b6d65341-1\n0020: 36 37 63 2d 34 31 64 66 2d 39 62 35 63 2d 34 31 67c-41df-9b5c-41\n0030: 63 64 65 64 39 39 63 32 32 39 04 68 6f 73 74 0d cded99c229.host.\n0040: 63 6f 6e 74 72 6f 6c 2d 70 6c 61 6e 65 05 6f 78 control-plane.ox\n0050: 69 64 65 08 69 6e 74 65 72 6e 61 6c 00 00 1c 00 ide.internal....\n0060: 01 00 00 29 04 d0 00 00 00 00 00 00 ...)........ kind = dns peer_addr = [::1]:56629 req_id = 73286138-f91a-4aa7-aa61-68453f5bfab0
166462025-04-30T06:44:29.611ZDEBGtest_utilization (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 61853,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n query_type: AAAA,\n query_class: IN,\n },\n },\n original: [\n 45,\n 100,\n 101,\n 110,\n 100,\n 114,\n 105,\n 116,\n 101,\n 45,\n 98,\n 54,\n 100,\n 54,\n 53,\n 51,\n 52,\n 49,\n 45,\n 49,\n 54,\n 55,\n 99,\n 45,\n 52,\n 49,\n 100,\n 102,\n 45,\n 57,\n 98,\n 53,\n 99,\n 45,\n 52,\n 49,\n 99,\n 100,\n 101,\n 100,\n 57,\n 57,\n 99,\n 50,\n 50,\n 57,\n 4,\n 104,\n 111,\n 115,\n 116,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 28,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:56629 req_id = 73286138-f91a-4aa7-aa61-68453f5bfab0
166472025-04-30T06:44:29.611ZDEBGtest_utilization (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
166482025-04-30T06:44:29.611ZDEBGtest_utilization (dns): dns response additional_records = [] kind = dns peer_addr = [::1]:56629 query = LowerQuery { name: LowerName(Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.")), original: Query { name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), query_type: AAAA, query_class: IN } } records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] req_id = 73286138-f91a-4aa7-aa61-68453f5bfab0
166492025-04-30T06:44:29.611ZTRACtest_utilization (dns): buffer buffer = Length: 72 (0x48) bytes\n0000: 34 e0 01 00 00 01 00 00 00 00 00 01 09 5f 64 65 4............_de\n0010: 6e 64 72 69 74 65 04 5f 74 63 70 0d 63 6f 6e 74 ndrite._tcp.cont\n0020: 72 6f 6c 2d 70 6c 61 6e 65 05 6f 78 69 64 65 08 rol-plane.oxide.\n0030: 69 6e 74 65 72 6e 61 6c 00 00 21 00 01 00 00 29 internal..!....)\n0040: 04 d0 00 00 00 00 00 00 ........ kind = dns peer_addr = [::1]:54622 req_id = 078c7f92-4c8d-45db-a7c1-78db158180fb
166502025-04-30T06:44:29.611ZDEBGtest_utilization (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 13536,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_dendrite._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_dendrite._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 9,\n 95,\n 100,\n 101,\n 110,\n 100,\n 114,\n 105,\n 116,\n 101,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:54622 req_id = 078c7f92-4c8d-45db-a7c1-78db158180fb
166512025-04-30T06:44:29.611ZDEBGtest_utilization (store): query key key = _dendrite._tcp
166522025-04-30T06:44:29.611ZDEBGtest_utilization (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
166532025-04-30T06:44:29.611ZDEBGtest_utilization (dns): dns response additional_records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal"), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] kind = dns peer_addr = [::1]:54622 query = LowerQuery { name: LowerName(Name("_dendrite._tcp.control-plane.oxide.internal.")), original: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN } } records = [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 47974, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal") })) }] req_id = 078c7f92-4c8d-45db-a7c1-78db158180fb
166542025-04-30T06:44:29.671ZTRACtest_utilization (DnsResolver): lookup_all_socket_v6 srv dns_name = _clickhouse-admin-keeper._tcp.control-plane.oxide.internal
166552025-04-30T06:44:29.672ZDEBGtest_utilization (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 47974, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }], valid_until: Instant { tv_sec: 4334, tv_nsec: 471280899 } })
166562025-04-30T06:44:29.672ZTRACtest_utilization (DnsResolver): lookup_service_targets: looking up SRV target
166572025-04-30T06:44:29.682ZTRACtest_utilization (dns): buffer buffer = Length: 87 (0x57) bytes\n0000: 88 55 01 00 00 01 00 00 00 00 00 01 18 5f 63 6c .U..........._cl\n0010: 69 63 6b 68 6f 75 73 65 2d 61 64 6d 69 6e 2d 6b ickhouse-admin-k\n0020: 65 65 70 65 72 04 5f 74 63 70 0d 63 6f 6e 74 72 eeper._tcp.contr\n0030: 6f 6c 2d 70 6c 61 6e 65 05 6f 78 69 64 65 08 69 ol-plane.oxide.i\n0040: 6e 74 65 72 6e 61 6c 00 00 21 00 01 00 00 29 04 nternal..!....).\n0050: d0 00 00 00 00 00 00 ....... kind = dns peer_addr = [::1]:57054 req_id = ddb15a02-6c81-4f5a-8bcd-219a36403171
166582025-04-30T06:44:29.682ZDEBGtest_utilization (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 34901,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 24,\n 95,\n 99,\n 108,\n 105,\n 99,\n 107,\n 104,\n 111,\n 117,\n 115,\n 101,\n 45,\n 97,\n 100,\n 109,\n 105,\n 110,\n 45,\n 107,\n 101,\n 101,\n 112,\n 101,\n 114,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:57054 req_id = ddb15a02-6c81-4f5a-8bcd-219a36403171
166592025-04-30T06:44:29.682ZDEBGtest_utilization (store): query key key = _clickhouse-admin-keeper._tcp
166602025-04-30T06:44:29.682ZERROtest_utilization (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 34901, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 1, }, query: WireQuery { query: LowerQuery { name: LowerName( Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), ), original: Query { name: Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN, }, }, original: [ 24, 95, 99, 108, 105, 99, 107, 104, 111, 117, 115, 101, 45, 97, 100, 109, 105, 110, 45, 107, 101, 101, 112, 101, 114, 4, 95, 116, 99, 112, 13, 99, 111, 110, 116, 114, 111, 108, 45, 112, 108, 97, 110, 101, 5, 111, 120, 105, 100, 101, 8, 105, 110, 116, 101, 114, 110, 97, 108, 0, 0, 33, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: Some( Edns { rcode_high: 0, version: 0, dnssec_ok: false, max_payload: 1232, options: OPT { options: {}, }, }, ), } NXDOMAIN: no records found for name: "_clickhouse-admin-keeper._tcp.control-plane.oxide.internal." kind = dns peer_addr = [::1]:57054 req_id = ddb15a02-6c81-4f5a-8bcd-219a36403171
166612025-04-30T06:44:29.683ZTRACtest_utilization (dns): buffer buffer = Length: 108 (0x6c) bytes\n0000: 67 3c 01 00 00 01 00 00 00 00 00 01 2d 64 65 6e g<..........-den\n0010: 64 72 69 74 65 2d 62 36 64 36 35 33 34 31 2d 31 drite-b6d65341-1\n0020: 36 37 63 2d 34 31 64 66 2d 39 62 35 63 2d 34 31 67c-41df-9b5c-41\n0030: 63 64 65 64 39 39 63 32 32 39 04 68 6f 73 74 0d cded99c229.host.\n0040: 63 6f 6e 74 72 6f 6c 2d 70 6c 61 6e 65 05 6f 78 control-plane.ox\n0050: 69 64 65 08 69 6e 74 65 72 6e 61 6c 00 00 1c 00 ide.internal....\n0060: 01 00 00 29 04 d0 00 00 00 00 00 00 ...)........ kind = dns peer_addr = [::1]:63963 req_id = 56e348b7-ecee-4c25-9cc4-55fd9cff2bad
166622025-04-30T06:44:29.684ZDEBGtest_utilization (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 26428,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n query_type: AAAA,\n query_class: IN,\n },\n },\n original: [\n 45,\n 100,\n 101,\n 110,\n 100,\n 114,\n 105,\n 116,\n 101,\n 45,\n 98,\n 54,\n 100,\n 54,\n 53,\n 51,\n 52,\n 49,\n 45,\n 49,\n 54,\n 55,\n 99,\n 45,\n 52,\n 49,\n 100,\n 102,\n 45,\n 57,\n 98,\n 53,\n 99,\n 45,\n 52,\n 49,\n 99,\n 100,\n 101,\n 100,\n 57,\n 57,\n 99,\n 50,\n 50,\n 57,\n 4,\n 104,\n 111,\n 115,\n 116,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 28,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:63963 req_id = 56e348b7-ecee-4c25-9cc4-55fd9cff2bad
166632025-04-30T06:44:29.686ZDEBGtest_utilization (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
166642025-04-30T06:44:29.686ZDEBGtest_utilization (dns): dns response additional_records = [] kind = dns peer_addr = [::1]:63963 query = LowerQuery { name: LowerName(Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.")), original: Query { name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), query_type: AAAA, query_class: IN } } records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] req_id = 56e348b7-ecee-4c25-9cc4-55fd9cff2bad
166652025-04-30T06:44:29.687ZDEBGtest_utilization (BackgroundTasks): begin collection background_task = inventory_collection
166662025-04-30T06:44:29.687ZDEBGtest_utilization (BackgroundTasks): begin collection from MGS background_task = inventory_collection mgs_url = http://[::1]:48759
166672025-04-30T06:44:29.687ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/ignition
166682025-04-30T06:44:29.688ZINFOtest_utilization (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager
166692025-04-30T06:44:29.756ZINFOtest_utilization (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager zone_address = ::1
166702025-04-30T06:44:29.757ZDEBGtest_utilization (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[::1]:12225/local/switch-id
166712025-04-30T06:44:29.757ZDEBGtest_utilization (MgsClient): client response background_task = bfd_manager result = Err(reqwest::Error { kind: Request, url: "http://[::1]:12225/local/switch-id", source: hyper_util::client::legacy::Error(Connect, ConnectError("tcp connect error", Os { code: 146, kind: ConnectionRefused, message: "Connection refused" })) })
166722025-04-30T06:44:29.757ZWARNtest_utilization (BackgroundTasks): failed to identify switch slot for dendrite background_task = bfd_manager reason = Communication Error: error sending request for url (http://[::1]:12225/local/switch-id): client error (Connect): tcp connect error: Connection refused (os error 146): Connection refused (os error 146) zone_address = ::1
166732025-04-30T06:44:29.759ZWARNtest_utilization (BackgroundTasks): Failed to map switch zone addr: Communication Error: error sending request for url (http://[::1]:12225/local/switch-id), retrying background_task = bfd_manager
166742025-04-30T06:44:29.770ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/ignition", status: 200, headers: {"content-type": "application/json", "x-request-id": "033aba9a-bb64-4bba-9d3a-f5c2ccf95859", "content-length": "835", "date": "Wed, 30 Apr 2025 06:44:28 GMT"} })
166752025-04-30T06:44:29.770ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0
166762025-04-30T06:44:29.779ZTRACtest_utilization (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal
166772025-04-30T06:44:29.796ZTRACtest_utilization (dns): buffer buffer = Length: 72 (0x48) bytes\n0000: 57 77 01 00 00 01 00 00 00 00 00 01 09 5f 64 65 Ww..........._de\n0010: 6e 64 72 69 74 65 04 5f 74 63 70 0d 63 6f 6e 74 ndrite._tcp.cont\n0020: 72 6f 6c 2d 70 6c 61 6e 65 05 6f 78 69 64 65 08 rol-plane.oxide.\n0030: 69 6e 74 65 72 6e 61 6c 00 00 21 00 01 00 00 29 internal..!....)\n0040: 04 d0 00 00 00 00 00 00 ........ kind = dns peer_addr = [::1]:62131 req_id = 6648d10b-60fc-4194-b0bd-497534c3ecb7
166782025-04-30T06:44:29.797ZDEBGtest_utilization (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 22391,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_dendrite._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_dendrite._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 9,\n 95,\n 100,\n 101,\n 110,\n 100,\n 114,\n 105,\n 116,\n 101,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:62131 req_id = 6648d10b-60fc-4194-b0bd-497534c3ecb7
166792025-04-30T06:44:29.799ZDEBGtest_utilization (store): query key key = _dendrite._tcp
166802025-04-30T06:44:29.800ZDEBGtest_utilization (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
166812025-04-30T06:44:29.801ZDEBGtest_utilization (dns): dns response additional_records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal"), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] kind = dns peer_addr = [::1]:62131 query = LowerQuery { name: LowerName(Name("_dendrite._tcp.control-plane.oxide.internal.")), original: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN } } records = [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 47974, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal") })) }] req_id = 6648d10b-60fc-4194-b0bd-497534c3ecb7
166822025-04-30T06:44:29.804ZDEBGtest_utilization (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 47974, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }], valid_until: Instant { tv_sec: 4334, tv_nsec: 603631468 } })
166832025-04-30T06:44:29.804ZTRACtest_utilization (DnsResolver): lookup_service_targets: looking up SRV target
166842025-04-30T06:44:29.805ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "9111e692-37f2-4a2c-b2f8-975487852585", "content-length": "734", "date": "Wed, 30 Apr 2025 06:44:28 GMT"} })
166852025-04-30T06:44:29.806ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0/component/sp/caboose?firmware_slot=0
166862025-04-30T06:44:29.810ZTRACtest_utilization (dns): buffer buffer = Length: 108 (0x6c) bytes\n0000: 94 90 01 00 00 01 00 00 00 00 00 01 2d 64 65 6e ............-den\n0010: 64 72 69 74 65 2d 62 36 64 36 35 33 34 31 2d 31 drite-b6d65341-1\n0020: 36 37 63 2d 34 31 64 66 2d 39 62 35 63 2d 34 31 67c-41df-9b5c-41\n0030: 63 64 65 64 39 39 63 32 32 39 04 68 6f 73 74 0d cded99c229.host.\n0040: 63 6f 6e 74 72 6f 6c 2d 70 6c 61 6e 65 05 6f 78 control-plane.ox\n0050: 69 64 65 08 69 6e 74 65 72 6e 61 6c 00 00 1c 00 ide.internal....\n0060: 01 00 00 29 04 d0 00 00 00 00 00 00 ...)........ kind = dns peer_addr = [::1]:61927 req_id = d275aa90-91ae-4a73-b8fb-76ecbcda752e
166872025-04-30T06:44:29.810ZDEBGtest_utilization (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 38032,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n query_type: AAAA,\n query_class: IN,\n },\n },\n original: [\n 45,\n 100,\n 101,\n 110,\n 100,\n 114,\n 105,\n 116,\n 101,\n 45,\n 98,\n 54,\n 100,\n 54,\n 53,\n 51,\n 52,\n 49,\n 45,\n 49,\n 54,\n 55,\n 99,\n 45,\n 52,\n 49,\n 100,\n 102,\n 45,\n 57,\n 98,\n 53,\n 99,\n 45,\n 52,\n 49,\n 99,\n 100,\n 101,\n 100,\n 57,\n 57,\n 99,\n 50,\n 50,\n 57,\n 4,\n 104,\n 111,\n 115,\n 116,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 28,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:61927 req_id = d275aa90-91ae-4a73-b8fb-76ecbcda752e
166882025-04-30T06:44:29.811ZDEBGtest_utilization (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
166892025-04-30T06:44:29.811ZDEBGtest_utilization (dns): dns response additional_records = [] kind = dns peer_addr = [::1]:61927 query = LowerQuery { name: LowerName(Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.")), original: Query { name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), query_type: AAAA, query_class: IN } } records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] req_id = d275aa90-91ae-4a73-b8fb-76ecbcda752e
166902025-04-30T06:44:29.811ZINFOtest_utilization (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager rack_id = c19a698f-c6f9-4a17-ae30-20d711b8f7dc
166912025-04-30T06:44:29.889ZINFOtest_utilization (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager rack_id = c19a698f-c6f9-4a17-ae30-20d711b8f7dc zone_address = ::1
166922025-04-30T06:44:29.889ZDEBGtest_utilization (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = c19a698f-c6f9-4a17-ae30-20d711b8f7dc uri = http://[::1]:12225/local/switch-id
166932025-04-30T06:44:29.889ZDEBGtest_utilization (MgsClient): client response background_task = switch_port_config_manager rack_id = c19a698f-c6f9-4a17-ae30-20d711b8f7dc result = Err(reqwest::Error { kind: Request, url: "http://[::1]:12225/local/switch-id", source: hyper_util::client::legacy::Error(Connect, ConnectError("tcp connect error", Os { code: 146, kind: ConnectionRefused, message: "Connection refused" })) })
166942025-04-30T06:44:29.889ZWARNtest_utilization (BackgroundTasks): failed to identify switch slot for dendrite background_task = switch_port_config_manager rack_id = c19a698f-c6f9-4a17-ae30-20d711b8f7dc reason = Communication Error: error sending request for url (http://[::1]:12225/local/switch-id): client error (Connect): tcp connect error: Connection refused (os error 146): Connection refused (os error 146) zone_address = ::1
166952025-04-30T06:44:29.890ZWARNtest_utilization (BackgroundTasks): Failed to map switch zone addr: Communication Error: error sending request for url (http://[::1]:12225/local/switch-id), retrying background_task = switch_port_config_manager rack_id = c19a698f-c6f9-4a17-ae30-20d711b8f7dc
166962025-04-30T06:44:29.894ZDEBGtest_utilization: attempt to fetch latest inventory collection result = Ok("found")
166972025-04-30T06:44:29.894ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "8096ebda-d5a3-41ed-a684-abfd026b2da5", "content-length": "111", "date": "Wed, 30 Apr 2025 06:44:28 GMT"} })
166982025-04-30T06:44:29.895ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0/component/sp/caboose?firmware_slot=1
166992025-04-30T06:44:29.898ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "2a02e8ba-f97a-42ee-b4b5-9192e9e7da3e", "content-length": "111", "date": "Wed, 30 Apr 2025 06:44:28 GMT"} })
167002025-04-30T06:44:29.898ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0/component/rot/caboose?firmware_slot=0
167012025-04-30T06:44:29.900ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "f749f6be-740a-4ef2-b7fb-eda94bc29d1d", "content-length": "170", "date": "Wed, 30 Apr 2025 06:44:28 GMT"} })
167022025-04-30T06:44:29.901ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0/component/rot/caboose?firmware_slot=1
167032025-04-30T06:44:29.903ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "c71d3266-64ed-497a-b053-99a9e9dccda3", "content-length": "170", "date": "Wed, 30 Apr 2025 06:44:28 GMT"} })
167042025-04-30T06:44:29.903ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0/component/stage0/caboose?firmware_slot=0
167052025-04-30T06:44:29.906ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "7a70a85a-612e-41e4-88fa-a0961390c885", "content-length": "179", "date": "Wed, 30 Apr 2025 06:44:28 GMT"} })
167062025-04-30T06:44:29.906ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0/component/stage0/caboose?firmware_slot=1
167072025-04-30T06:44:29.909ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "8b19f408-9a5d-4a95-8883-898d0c88d65b", "content-length": "179", "date": "Wed, 30 Apr 2025 06:44:28 GMT"} })
167082025-04-30T06:44:29.909ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0/component/rot/cmpa
167092025-04-30T06:44:29.910ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "48d6350c-1e80-4c8c-a3f8-f25bf6814d0e", "content-length": "702", "date": "Wed, 30 Apr 2025 06:44:28 GMT"} })
167102025-04-30T06:44:29.911ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0/component/rot/cfpa
167112025-04-30T06:44:29.912ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "38ff41bb-caf0-4753-9673-9e18e5d86791", "content-length": "727", "date": "Wed, 30 Apr 2025 06:44:28 GMT"} })
167122025-04-30T06:44:29.912ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0/component/rot/cfpa
167132025-04-30T06:44:29.913ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "19d3654c-54d7-45d6-9c88-fc060eef9510", "content-length": "729", "date": "Wed, 30 Apr 2025 06:44:28 GMT"} })
167142025-04-30T06:44:29.913ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0/component/rot/cfpa
167152025-04-30T06:44:29.914ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "d4ab90c7-6b27-4386-9c93-18292923f4b9", "content-length": "728", "date": "Wed, 30 Apr 2025 06:44:28 GMT"} })
167162025-04-30T06:44:29.915ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1
167172025-04-30T06:44:29.916ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1", status: 200, headers: {"content-type": "application/json", "x-request-id": "c7b889f8-56e3-4ca9-8444-f029a6ef7b0e", "content-length": "734", "date": "Wed, 30 Apr 2025 06:44:28 GMT"} })
167182025-04-30T06:44:29.918ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1/component/sp/caboose?firmware_slot=0
167192025-04-30T06:44:29.921ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "d6896f4c-7e38-4990-ad68-6282a8bab7d0", "content-length": "111", "date": "Wed, 30 Apr 2025 06:44:28 GMT"} })
167202025-04-30T06:44:29.921ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1/component/sp/caboose?firmware_slot=1
167212025-04-30T06:44:29.926ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "49c79c20-0ab8-4dca-bf5d-696084510de0", "content-length": "111", "date": "Wed, 30 Apr 2025 06:44:28 GMT"} })
167222025-04-30T06:44:29.926ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1/component/rot/caboose?firmware_slot=0
167232025-04-30T06:44:29.929ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "0d23e1f9-946d-492f-993a-a876513d17c9", "content-length": "170", "date": "Wed, 30 Apr 2025 06:44:28 GMT"} })
167242025-04-30T06:44:29.930ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1/component/rot/caboose?firmware_slot=1
167252025-04-30T06:44:29.931ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "7cfb6b84-8ac9-4ec1-9641-c4dddfcd8245", "content-length": "170", "date": "Wed, 30 Apr 2025 06:44:28 GMT"} })
167262025-04-30T06:44:29.931ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1/component/stage0/caboose?firmware_slot=0
167272025-04-30T06:44:29.934ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "7d2e97d1-be99-4913-bfcb-5353e7ebdd13", "content-length": "179", "date": "Wed, 30 Apr 2025 06:44:28 GMT"} })
167282025-04-30T06:44:29.935ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1/component/stage0/caboose?firmware_slot=1
167292025-04-30T06:44:29.939ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "826e76d6-1662-4001-bf49-5e544add8346", "content-length": "179", "date": "Wed, 30 Apr 2025 06:44:28 GMT"} })
167302025-04-30T06:44:29.939ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1/component/rot/cmpa
167312025-04-30T06:44:29.940ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "c5a9c281-445c-4288-af1c-e1ac96bf1564", "content-length": "702", "date": "Wed, 30 Apr 2025 06:44:28 GMT"} })
167322025-04-30T06:44:29.941ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1/component/rot/cfpa
167332025-04-30T06:44:29.942ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "d820a265-7a8d-420f-ad4a-01c1d95e6193", "content-length": "727", "date": "Wed, 30 Apr 2025 06:44:28 GMT"} })
167342025-04-30T06:44:29.942ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1/component/rot/cfpa
167352025-04-30T06:44:29.944ZTRACtest_utilization (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory
167362025-04-30T06:44:29.944ZTRACtest_utilization (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
167372025-04-30T06:44:29.944ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
167382025-04-30T06:44:29.945ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
167392025-04-30T06:44:29.945ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
167402025-04-30T06:44:29.946ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "fef0cb8d-7719-4de4-bf77-bfc26e68bf54", "content-length": "729", "date": "Wed, 30 Apr 2025 06:44:28 GMT"} })
167412025-04-30T06:44:29.946ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1/component/rot/cfpa
167422025-04-30T06:44:29.947ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "1998ba6d-29a6-4763-a657-f1a4728a7ccd", "content-length": "728", "date": "Wed, 30 Apr 2025 06:44:28 GMT"} })
167432025-04-30T06:44:29.947ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0
167442025-04-30T06:44:29.949ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
167452025-04-30T06:44:29.956ZDEBGtest_utilization (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory result = Ok(())
167462025-04-30T06:44:29.956ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
167472025-04-30T06:44:29.956ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
167482025-04-30T06:44:29.957ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
167492025-04-30T06:44:29.957ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "55f1ec97-eacb-4dbb-821d-a4456cbe6a8c", "content-length": "723", "date": "Wed, 30 Apr 2025 06:44:28 GMT"} })
167502025-04-30T06:44:29.958ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0/component/sp/caboose?firmware_slot=0
167512025-04-30T06:44:29.960ZINFOtest_utilization (oximeter-agent): refreshing list of producers from Nexus collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 file = oximeter/collector/src/agent.rs:562
167522025-04-30T06:44:29.961ZDEBGtest_utilization (oximeter): client request body = None method = GET uri = http://127.0.0.1:32956/metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?sort_by=id_ascending
167532025-04-30T06:44:29.962ZTRACtest_utilization (dropshot_internal): incoming request local_addr = 127.0.0.1:32956 method = GET remote_addr = 127.0.0.1:55797 req_id = ba9c8874-c347-46ec-be5b-683983ac7044 uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?sort_by=id_ascending
167542025-04-30T06:44:29.962ZTRACtest_utilization (dropshot_internal): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = 127.0.0.1:32956 method = GET remote_addr = 127.0.0.1:55797 req_id = ba9c8874-c347-46ec-be5b-683983ac7044 resource = Database uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?sort_by=id_ascending
167552025-04-30T06:44:29.962ZDEBGtest_utilization (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = 127.0.0.1:32956 method = GET remote_addr = 127.0.0.1:55797 req_id = ba9c8874-c347-46ec-be5b-683983ac7044 roles = RoleSet { roles: {} } uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?sort_by=id_ascending
167562025-04-30T06:44:29.963ZDEBGtest_utilization (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = 127.0.0.1:32956 method = GET remote_addr = 127.0.0.1:55797 req_id = ba9c8874-c347-46ec-be5b-683983ac7044 resource = Database result = Ok(()) uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?sort_by=id_ascending
167572025-04-30T06:44:29.964ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
167582025-04-30T06:44:29.964ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
167592025-04-30T06:44:29.964ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
167602025-04-30T06:44:29.968ZINFOtest_utilization (dropshot_internal): request completed latency_us = 7085 local_addr = 127.0.0.1:32956 method = GET remote_addr = 127.0.0.1:55797 req_id = ba9c8874-c347-46ec-be5b-683983ac7044 response_code = 200 uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?sort_by=id_ascending
167612025-04-30T06:44:29.971ZDEBGtest_utilization (oximeter): client response result = Ok(Response { url: "http://127.0.0.1:32956/metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?sort_by=id_ascending", status: 200, headers: {"content-type": "application/json", "x-request-id": "ba9c8874-c347-46ec-be5b-683983ac7044", "content-length": "405", "date": "Wed, 30 Apr 2025 06:44:28 GMT"} })
167622025-04-30T06:44:29.971ZDEBGtest_utilization (oximeter): client request body = None method = GET uri = http://127.0.0.1:32956/metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJhYzU1ZDhkOS1hMjk4LTQ1MzQtODk4Yi1iZWVlMWM5ZTRlNTIifX0%3D
167632025-04-30T06:44:29.972ZTRACtest_utilization (dropshot_internal): incoming request local_addr = 127.0.0.1:32956 method = GET remote_addr = 127.0.0.1:55797 req_id = debd6e1c-e9ee-430b-96f9-a345736fa8c3 uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJhYzU1ZDhkOS1hMjk4LTQ1MzQtODk4Yi1iZWVlMWM5ZTRlNTIifX0%3D
167642025-04-30T06:44:29.972ZTRACtest_utilization (dropshot_internal): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = 127.0.0.1:32956 method = GET remote_addr = 127.0.0.1:55797 req_id = debd6e1c-e9ee-430b-96f9-a345736fa8c3 resource = Database uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJhYzU1ZDhkOS1hMjk4LTQ1MzQtODk4Yi1iZWVlMWM5ZTRlNTIifX0%3D
167652025-04-30T06:44:29.972ZDEBGtest_utilization (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = 127.0.0.1:32956 method = GET remote_addr = 127.0.0.1:55797 req_id = debd6e1c-e9ee-430b-96f9-a345736fa8c3 roles = RoleSet { roles: {} } uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJhYzU1ZDhkOS1hMjk4LTQ1MzQtODk4Yi1iZWVlMWM5ZTRlNTIifX0%3D
167662025-04-30T06:44:29.972ZDEBGtest_utilization (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = 127.0.0.1:32956 method = GET remote_addr = 127.0.0.1:55797 req_id = debd6e1c-e9ee-430b-96f9-a345736fa8c3 resource = Database result = Ok(()) uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJhYzU1ZDhkOS1hMjk4LTQ1MzQtODk4Yi1iZWVlMWM5ZTRlNTIifX0%3D
167672025-04-30T06:44:29.974ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "c236f63f-836a-482f-8da2-c3803fb886f4", "content-length": "109", "date": "Wed, 30 Apr 2025 06:44:28 GMT"} })
167682025-04-30T06:44:29.974ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0/component/sp/caboose?firmware_slot=1
167692025-04-30T06:44:29.976ZINFOtest_utilization (dropshot_internal): request completed latency_us = 4140 local_addr = 127.0.0.1:32956 method = GET remote_addr = 127.0.0.1:55797 req_id = debd6e1c-e9ee-430b-96f9-a345736fa8c3 response_code = 200 uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJhYzU1ZDhkOS1hMjk4LTQ1MzQtODk4Yi1iZWVlMWM5ZTRlNTIifX0%3D
167702025-04-30T06:44:29.977ZDEBGtest_utilization (oximeter): client response result = Ok(Response { url: "http://127.0.0.1:32956/metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJhYzU1ZDhkOS1hMjk4LTQ1MzQtODk4Yi1iZWVlMWM5ZTRlNTIifX0%3D", status: 200, headers: {"content-type": "application/json", "x-request-id": "debd6e1c-e9ee-430b-96f9-a345736fa8c3", "content-length": "29", "date": "Wed, 30 Apr 2025 06:44:28 GMT"} })
167712025-04-30T06:44:29.978ZTRACtest_utilization (oximeter-agent): ignoring request to update existing metric producer, since the endpoint information is the same as the existing collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
167722025-04-30T06:44:29.978ZTRACtest_utilization (oximeter-agent): ignoring request to update existing metric producer, since the endpoint information is the same as the existing collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = ac55d8d9-a298-4534-898b-beee1c9e4e52
167732025-04-30T06:44:29.979ZINFOtest_utilization (oximeter-agent): refreshed list of producers from Nexus collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 file = oximeter/collector/src/agent.rs:637 n_current_tasks = 2 n_pruned_tasks = 0
167742025-04-30T06:44:29.983ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "536971f6-d603-4e10-a601-aeb987dfea75", "content-length": "109", "date": "Wed, 30 Apr 2025 06:44:28 GMT"} })
167752025-04-30T06:44:29.984ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0/component/rot/caboose?firmware_slot=0
167762025-04-30T06:44:29.987ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "3b234a2d-9f1d-4606-8fec-d351163dadf8", "content-length": "169", "date": "Wed, 30 Apr 2025 06:44:28 GMT"} })
167772025-04-30T06:44:29.987ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0/component/rot/caboose?firmware_slot=1
167782025-04-30T06:44:29.991ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "a023b031-117c-44fd-b53c-f5b6b4fe3ebb", "content-length": "169", "date": "Wed, 30 Apr 2025 06:44:29 GMT"} })
167792025-04-30T06:44:29.991ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0/component/stage0/caboose?firmware_slot=0
167802025-04-30T06:44:29.994ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "42fa83c2-09a1-456b-a6f3-1eb96b7db308", "content-length": "178", "date": "Wed, 30 Apr 2025 06:44:29 GMT"} })
167812025-04-30T06:44:29.995ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0/component/stage0/caboose?firmware_slot=1
167822025-04-30T06:44:29.996ZDEBGtest_utilization (collection-task): sent timer-based collection request to the collection task collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
167832025-04-30T06:44:29.996ZDEBGtest_utilization (collection-task): collecting from producer collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
167842025-04-30T06:44:29.999ZTRACtest_utilization (collection-task): sent collection request to producer collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
167852025-04-30T06:44:30.002ZDEBGtest_utilization (collection-task): collected results from producer collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 n_results = 1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
167862025-04-30T06:44:30.002ZTRACtest_utilization (collection-task): forwarded results to main collection loop collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
167872025-04-30T06:44:30.002ZTRACtest_utilization (collection-task): top of inner collection loop, waiting for next request collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
167882025-04-30T06:44:30.006ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "a66f18fd-82c9-49a6-bd8f-6e05c9c3a172", "content-length": "178", "date": "Wed, 30 Apr 2025 06:44:29 GMT"} })
167892025-04-30T06:44:30.006ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0/component/rot/cmpa
167902025-04-30T06:44:30.009ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "d8ff32df-58ce-4ecb-bee6-199c9059bbef", "content-length": "702", "date": "Wed, 30 Apr 2025 06:44:29 GMT"} })
167912025-04-30T06:44:30.009ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0/component/rot/cfpa
167922025-04-30T06:44:30.010ZDEBGtest_utilization: attempt to fetch latest inventory collection result = Ok("found")
167932025-04-30T06:44:30.011ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "078d98da-4547-442e-b987-d02b0a3acb0e", "content-length": "727", "date": "Wed, 30 Apr 2025 06:44:29 GMT"} })
167942025-04-30T06:44:30.012ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0/component/rot/cfpa
167952025-04-30T06:44:30.013ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "f9d6e357-eb70-434c-beba-fb9023e33c0a", "content-length": "729", "date": "Wed, 30 Apr 2025 06:44:29 GMT"} })
167962025-04-30T06:44:30.013ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0/component/rot/cfpa
167972025-04-30T06:44:30.014ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "b1b46326-b168-4fde-b609-50fa651d8ecb", "content-length": "728", "date": "Wed, 30 Apr 2025 06:44:29 GMT"} })
167982025-04-30T06:44:30.014ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1
167992025-04-30T06:44:30.015ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1", status: 200, headers: {"content-type": "application/json", "x-request-id": "87c18389-9b6f-47b8-ba0f-d63d8fa112f8", "content-length": "723", "date": "Wed, 30 Apr 2025 06:44:29 GMT"} })
168002025-04-30T06:44:30.015ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1/component/sp/caboose?firmware_slot=0
168012025-04-30T06:44:30.017ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "30eab5c3-9799-4831-8f7c-d736eaec97a7", "content-length": "109", "date": "Wed, 30 Apr 2025 06:44:29 GMT"} })
168022025-04-30T06:44:30.017ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1/component/sp/caboose?firmware_slot=1
168032025-04-30T06:44:30.019ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "c3238cf0-9c22-4a89-9b85-733b8973ecee", "content-length": "109", "date": "Wed, 30 Apr 2025 06:44:29 GMT"} })
168042025-04-30T06:44:30.019ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1/component/rot/caboose?firmware_slot=0
168052025-04-30T06:44:30.022ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "55d88895-a204-4a63-a865-efa7be78db9f", "content-length": "169", "date": "Wed, 30 Apr 2025 06:44:29 GMT"} })
168062025-04-30T06:44:30.022ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1/component/rot/caboose?firmware_slot=1
168072025-04-30T06:44:30.024ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "58b77fa6-c636-42d1-b1e1-e46d418ebdd5", "content-length": "169", "date": "Wed, 30 Apr 2025 06:44:29 GMT"} })
168082025-04-30T06:44:30.025ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1/component/stage0/caboose?firmware_slot=0
168092025-04-30T06:44:30.027ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "1de3cb96-cb3f-4cf8-ab29-9cba24dadc73", "content-length": "178", "date": "Wed, 30 Apr 2025 06:44:29 GMT"} })
168102025-04-30T06:44:30.027ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1/component/stage0/caboose?firmware_slot=1
168112025-04-30T06:44:30.029ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "71de3986-6ee3-46e6-913e-1e53afb1a501", "content-length": "178", "date": "Wed, 30 Apr 2025 06:44:29 GMT"} })
168122025-04-30T06:44:30.029ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1/component/rot/cmpa
168132025-04-30T06:44:30.031ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "61f3d3d9-1670-4e17-a808-9c7010cc2d23", "content-length": "702", "date": "Wed, 30 Apr 2025 06:44:29 GMT"} })
168142025-04-30T06:44:30.031ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1/component/rot/cfpa
168152025-04-30T06:44:30.032ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "40aed3b6-43dc-482e-a84d-d991c3b6458b", "content-length": "727", "date": "Wed, 30 Apr 2025 06:44:29 GMT"} })
168162025-04-30T06:44:30.033ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1/component/rot/cfpa
168172025-04-30T06:44:30.034ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "370f4509-6d06-4038-8682-4ac0b4b95a07", "content-length": "729", "date": "Wed, 30 Apr 2025 06:44:29 GMT"} })
168182025-04-30T06:44:30.034ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1/component/rot/cfpa
168192025-04-30T06:44:30.035ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "8a022208-4f61-48a3-9798-6cbc298cb5ae", "content-length": "728", "date": "Wed, 30 Apr 2025 06:44:29 GMT"} })
168202025-04-30T06:44:30.035ZTRACtest_utilization (BackgroundTasks): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet
168212025-04-30T06:44:30.035ZTRACtest_utilization (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
168222025-04-30T06:44:30.036ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
168232025-04-30T06:44:30.036ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
168242025-04-30T06:44:30.036ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
168252025-04-30T06:44:30.039ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
168262025-04-30T06:44:30.045ZDEBGtest_utilization (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet result = Ok(())
168272025-04-30T06:44:30.045ZTRACtest_utilization (BackgroundTasks): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet
168282025-04-30T06:44:30.045ZTRACtest_utilization (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
168292025-04-30T06:44:30.045ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
168302025-04-30T06:44:30.045ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
168312025-04-30T06:44:30.047ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
168322025-04-30T06:44:30.050ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
168332025-04-30T06:44:30.055ZDEBGtest_utilization (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet result = Ok(())
168342025-04-30T06:44:30.055ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
168352025-04-30T06:44:30.055ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
168362025-04-30T06:44:30.057ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
168372025-04-30T06:44:30.071ZTRACtest_utilization (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory
168382025-04-30T06:44:30.074ZTRACtest_utilization (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
168392025-04-30T06:44:30.074ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
168402025-04-30T06:44:30.075ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
168412025-04-30T06:44:30.075ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
168422025-04-30T06:44:30.144ZDEBGtest_utilization (BackgroundTasks): begin collection from Sled Agent background_task = inventory_collection sled_agent_url = http://[::1]:39365
168432025-04-30T06:44:30.144ZDEBGtest_utilization (BackgroundTasks): client request SledAgent = http://[::1]:39365 background_task = inventory_collection body = None method = GET uri = http://[::1]:39365/inventory
168442025-04-30T06:44:30.146ZINFOtest_utilization (dropshot): accepted connection local_addr = [::1]:39365 remote_addr = [::1]:37640 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
168452025-04-30T06:44:30.147ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
168462025-04-30T06:44:30.159ZDEBGtest_utilization (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory result = Ok(())
168472025-04-30T06:44:30.159ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
168482025-04-30T06:44:30.159ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
168492025-04-30T06:44:30.160ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
168502025-04-30T06:44:30.160ZTRACtest_utilization (dropshot): incoming request local_addr = [::1]:39365 method = GET remote_addr = [::1]:37640 req_id = 40ce6854-bf96-4613-927f-c8479747305c sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /inventory
168512025-04-30T06:44:30.162ZINFOtest_utilization (dropshot): request completed latency_us = 2063 local_addr = [::1]:39365 method = GET remote_addr = [::1]:37640 req_id = 40ce6854-bf96-4613-927f-c8479747305c response_code = 200 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /inventory
168522025-04-30T06:44:30.163ZDEBGtest_utilization (BackgroundTasks): client response SledAgent = http://[::1]:39365 background_task = inventory_collection result = Ok(Response { url: "http://[::1]:39365/inventory", status: 200, headers: {"content-type": "application/json", "x-request-id": "40ce6854-bf96-4613-927f-c8479747305c", "content-length": "3141", "date": "Wed, 30 Apr 2025 06:44:29 GMT"} })
168532025-04-30T06:44:30.165ZDEBGtest_utilization (BackgroundTasks): begin collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
168542025-04-30T06:44:30.165ZDEBGtest_utilization (BackgroundTasks): end collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
168552025-04-30T06:44:30.165ZDEBGtest_utilization (BackgroundTasks): finished collection background_task = inventory_collection
168562025-04-30T06:44:30.170ZTRACtest_utilization (BackgroundTasks): authorize begin action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory
168572025-04-30T06:44:30.170ZTRACtest_utilization (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
168582025-04-30T06:44:30.170ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
168592025-04-30T06:44:30.170ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
168602025-04-30T06:44:30.172ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
168612025-04-30T06:44:30.172ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
168622025-04-30T06:44:30.173ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
168632025-04-30T06:44:30.175ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
168642025-04-30T06:44:30.177ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
168652025-04-30T06:44:30.182ZDEBGtest_utilization (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory result = Ok(())
168662025-04-30T06:44:30.182ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
168672025-04-30T06:44:30.182ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
168682025-04-30T06:44:30.185ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
168692025-04-30T06:44:30.251ZWARNtest_utilization (datastore): Non-retryable transaction failure err = DatabaseError(UniqueViolation, "duplicate key value violates unique constraint \\"inv_caboose_pkey\\"") transaction = inventory_insert_collection
168702025-04-30T06:44:30.254ZWARNtest_utilization (BackgroundTasks): inventory collection failed background_task = inventory_collection error = failed to collect inventory: saving inventory to database: Internal Error: unexpected database error: duplicate key value violates unique constraint "inv_caboose_pkey"
168712025-04-30T06:44:30.254ZDEBGtest_utilization (BackgroundTasks): activation complete background_task = inventory_collection elapsed = 777.585418ms iteration = 49 status = {"error":"failed to collect inventory: saving inventory to database: Internal Error: unexpected database error: duplicate key value violates unique constraint \\"inv_caboose_pkey\\""}
168722025-04-30T06:44:30.254ZDEBGtest_utilization (BackgroundTasks): activating background_task = inventory_collection iteration = 50 reason = Signaled
168732025-04-30T06:44:30.254ZTRACtest_utilization (BackgroundTasks): authorize begin action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory
168742025-04-30T06:44:30.254ZTRACtest_utilization (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
168752025-04-30T06:44:30.254ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
168762025-04-30T06:44:30.254ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
168772025-04-30T06:44:30.255ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
168782025-04-30T06:44:30.261ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
168792025-04-30T06:44:30.271ZDEBGtest_utilization (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory result = Ok(())
168802025-04-30T06:44:30.271ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
168812025-04-30T06:44:30.271ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
168822025-04-30T06:44:30.277ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
168832025-04-30T06:44:30.278ZDEBGtest_utilization: attempt to fetch latest inventory collection result = Ok("found")
168842025-04-30T06:44:30.285ZDEBGtest_utilization (BackgroundTasks): inventory_prune_one: nothing eligible for removal (too few) background_task = inventory_collection candidates = [(13d8b8d9-6c6b-47a6-9190-952544edb75b, 0)]
168852025-04-30T06:44:30.285ZTRACtest_utilization (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal
168862025-04-30T06:44:30.286ZTRACtest_utilization (dns): buffer buffer = Length: 67 (0x43) bytes\n0000: 20 97 01 00 00 01 00 00 00 00 00 01 04 5f 6d 67 ............_mg\n0010: 73 04 5f 74 63 70 0d 63 6f 6e 74 72 6f 6c 2d 70 s._tcp.control-p\n0020: 6c 61 6e 65 05 6f 78 69 64 65 08 69 6e 74 65 72 lane.oxide.inter\n0030: 6e 61 6c 00 00 21 00 01 00 00 29 04 d0 00 00 00 nal..!....).....\n0040: 00 00 00 ... kind = dns peer_addr = [::1]:58202 req_id = 7c4fe8d1-7eeb-4344-b602-0a295e19aba1
168872025-04-30T06:44:30.286ZDEBGtest_utilization (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 8343,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_mgs._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_mgs._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 4,\n 95,\n 109,\n 103,\n 115,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:58202 req_id = 7c4fe8d1-7eeb-4344-b602-0a295e19aba1
168882025-04-30T06:44:30.288ZDEBGtest_utilization (store): query key key = _mgs._tcp
168892025-04-30T06:44:30.288ZDEBGtest_utilization (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
168902025-04-30T06:44:30.289ZDEBGtest_utilization (dns): dns response additional_records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal"), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] kind = dns peer_addr = [::1]:58202 query = LowerQuery { name: LowerName(Name("_mgs._tcp.control-plane.oxide.internal.")), original: Query { name: Name("_mgs._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN } } records = [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 48759, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal") })) }] req_id = 7c4fe8d1-7eeb-4344-b602-0a295e19aba1
168912025-04-30T06:44:30.289ZDEBGtest_utilization (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgs._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 48759, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }], valid_until: Instant { tv_sec: 4335, tv_nsec: 88491851 } })
168922025-04-30T06:44:30.289ZTRACtest_utilization (DnsResolver): lookup_service_targets: looking up SRV target
168932025-04-30T06:44:30.290ZTRACtest_utilization (dns): buffer buffer = Length: 108 (0x6c) bytes\n0000: 84 01 01 00 00 01 00 00 00 00 00 01 2d 64 65 6e ............-den\n0010: 64 72 69 74 65 2d 62 36 64 36 35 33 34 31 2d 31 drite-b6d65341-1\n0020: 36 37 63 2d 34 31 64 66 2d 39 62 35 63 2d 34 31 67c-41df-9b5c-41\n0030: 63 64 65 64 39 39 63 32 32 39 04 68 6f 73 74 0d cded99c229.host.\n0040: 63 6f 6e 74 72 6f 6c 2d 70 6c 61 6e 65 05 6f 78 control-plane.ox\n0050: 69 64 65 08 69 6e 74 65 72 6e 61 6c 00 00 1c 00 ide.internal....\n0060: 01 00 00 29 04 d0 00 00 00 00 00 00 ...)........ kind = dns peer_addr = [::1]:58846 req_id = 53c7f348-fe46-4931-9fe3-c87edae515d3
168942025-04-30T06:44:30.290ZDEBGtest_utilization (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 33793,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n query_type: AAAA,\n query_class: IN,\n },\n },\n original: [\n 45,\n 100,\n 101,\n 110,\n 100,\n 114,\n 105,\n 116,\n 101,\n 45,\n 98,\n 54,\n 100,\n 54,\n 53,\n 51,\n 52,\n 49,\n 45,\n 49,\n 54,\n 55,\n 99,\n 45,\n 52,\n 49,\n 100,\n 102,\n 45,\n 57,\n 98,\n 53,\n 99,\n 45,\n 52,\n 49,\n 99,\n 100,\n 101,\n 100,\n 57,\n 57,\n 99,\n 50,\n 50,\n 57,\n 4,\n 104,\n 111,\n 115,\n 116,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 28,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:58846 req_id = 53c7f348-fe46-4931-9fe3-c87edae515d3
168952025-04-30T06:44:30.290ZDEBGtest_utilization (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
168962025-04-30T06:44:30.291ZDEBGtest_utilization (dns): dns response additional_records = [] kind = dns peer_addr = [::1]:58846 query = LowerQuery { name: LowerName(Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.")), original: Query { name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), query_type: AAAA, query_class: IN } } records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] req_id = 53c7f348-fe46-4931-9fe3-c87edae515d3
168972025-04-30T06:44:30.369ZTRACtest_utilization (DnsResolver): lookup_all_socket_v6 srv dns_name = _clickhouse-admin-keeper._tcp.control-plane.oxide.internal
168982025-04-30T06:44:30.374ZTRACtest_utilization (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory
168992025-04-30T06:44:30.374ZTRACtest_utilization (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
169002025-04-30T06:44:30.374ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
169012025-04-30T06:44:30.374ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
169022025-04-30T06:44:30.376ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
169032025-04-30T06:44:30.376ZDEBGtest_utilization (results-sink): inserting 1 samples into database collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
169042025-04-30T06:44:30.376ZDEBGtest_utilization (results-sink-cluster): inserting 1 samples into database collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
169052025-04-30T06:44:30.377ZDEBGtest_utilization (clickhouse-client): unrolling 1 total samples collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 4d1485f0-110e-4161-9a33-c4610d68be5d
169062025-04-30T06:44:30.378ZTRACtest_utilization (clickhouse-client): inserting data collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 4d1485f0-110e-4161-9a33-c4610d68be5d n_columns = 4 n_rows = 2 sql = INSERT INTO oximeter.fields_string FORMAT Native
169072025-04-30T06:44:30.383ZTRACtest_utilization (dns): buffer buffer = Length: 87 (0x57) bytes\n0000: af dd 01 00 00 01 00 00 00 00 00 01 18 5f 63 6c ............._cl\n0010: 69 63 6b 68 6f 75 73 65 2d 61 64 6d 69 6e 2d 6b ickhouse-admin-k\n0020: 65 65 70 65 72 04 5f 74 63 70 0d 63 6f 6e 74 72 eeper._tcp.contr\n0030: 6f 6c 2d 70 6c 61 6e 65 05 6f 78 69 64 65 08 69 ol-plane.oxide.i\n0040: 6e 74 65 72 6e 61 6c 00 00 21 00 01 00 00 29 04 nternal..!....).\n0050: d0 00 00 00 00 00 00 ....... kind = dns peer_addr = [::1]:59747 req_id = 2fd4b0df-d5ec-43bc-bf38-d4289dc1efc5
169082025-04-30T06:44:30.383ZDEBGtest_utilization (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 45021,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 24,\n 95,\n 99,\n 108,\n 105,\n 99,\n 107,\n 104,\n 111,\n 117,\n 115,\n 101,\n 45,\n 97,\n 100,\n 109,\n 105,\n 110,\n 45,\n 107,\n 101,\n 101,\n 112,\n 101,\n 114,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:59747 req_id = 2fd4b0df-d5ec-43bc-bf38-d4289dc1efc5
169092025-04-30T06:44:30.383ZDEBGtest_utilization (store): query key key = _clickhouse-admin-keeper._tcp
169102025-04-30T06:44:30.384ZERROtest_utilization (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 45021, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 1, }, query: WireQuery { query: LowerQuery { name: LowerName( Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), ), original: Query { name: Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN, }, }, original: [ 24, 95, 99, 108, 105, 99, 107, 104, 111, 117, 115, 101, 45, 97, 100, 109, 105, 110, 45, 107, 101, 101, 112, 101, 114, 4, 95, 116, 99, 112, 13, 99, 111, 110, 116, 114, 111, 108, 45, 112, 108, 97, 110, 101, 5, 111, 120, 105, 100, 101, 8, 105, 110, 116, 101, 114, 110, 97, 108, 0, 0, 33, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: Some( Edns { rcode_high: 0, version: 0, dnssec_ok: false, max_payload: 1232, options: OPT { options: {}, }, }, ), } NXDOMAIN: no records found for name: "_clickhouse-admin-keeper._tcp.control-plane.oxide.internal." kind = dns peer_addr = [::1]:59747 req_id = 2fd4b0df-d5ec-43bc-bf38-d4289dc1efc5
169112025-04-30T06:44:30.386ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
169122025-04-30T06:44:30.403ZDEBGtest_utilization (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory result = Ok(())
169132025-04-30T06:44:30.403ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
169142025-04-30T06:44:30.403ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
169152025-04-30T06:44:30.403ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
169162025-04-30T06:44:30.404ZDEBGtest_utilization (BackgroundTasks): begin collection background_task = inventory_collection
169172025-04-30T06:44:30.404ZDEBGtest_utilization (BackgroundTasks): begin collection from MGS background_task = inventory_collection mgs_url = http://[::1]:48759
169182025-04-30T06:44:30.404ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/ignition
169192025-04-30T06:44:30.405ZDEBGtest_utilization (clickhouse-client): inserted rows into table collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 4d1485f0-110e-4161-9a33-c4610d68be5d n_rows = 2 table_name = fields_string
169202025-04-30T06:44:30.405ZTRACtest_utilization (clickhouse-client): inserting data collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 4d1485f0-110e-4161-9a33-c4610d68be5d n_columns = 4 n_rows = 1 sql = INSERT INTO oximeter.measurements_i64 FORMAT Native
169212025-04-30T06:44:30.410ZDEBGtest_utilization (clickhouse-client): inserted rows into table collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 4d1485f0-110e-4161-9a33-c4610d68be5d n_rows = 1 table_name = measurements_i64
169222025-04-30T06:44:30.410ZTRACtest_utilization (results-sink): successfully inserted samples collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
169232025-04-30T06:44:30.411ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
169242025-04-30T06:44:30.411ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
169252025-04-30T06:44:30.411ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
169262025-04-30T06:44:30.412ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/ignition", status: 200, headers: {"content-type": "application/json", "x-request-id": "217544b2-7050-47fc-8b9e-67b749a8d948", "content-length": "835", "date": "Wed, 30 Apr 2025 06:44:29 GMT"} })
169272025-04-30T06:44:30.412ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0
169282025-04-30T06:44:30.414ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "b96c38ff-8ec5-426b-a1d2-6e1eb7ecddfe", "content-length": "734", "date": "Wed, 30 Apr 2025 06:44:29 GMT"} })
169292025-04-30T06:44:30.414ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0/component/sp/caboose?firmware_slot=0
169302025-04-30T06:44:30.416ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "02891c4b-c060-47d7-a273-a23c0ca1fab1", "content-length": "111", "date": "Wed, 30 Apr 2025 06:44:29 GMT"} })
169312025-04-30T06:44:30.416ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0/component/sp/caboose?firmware_slot=1
169322025-04-30T06:44:30.428ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "6f669da7-3f09-4026-bb4f-2523e1968dab", "content-length": "111", "date": "Wed, 30 Apr 2025 06:44:29 GMT"} })
169332025-04-30T06:44:30.428ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0/component/rot/caboose?firmware_slot=0
169342025-04-30T06:44:30.433ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "dff391e0-718e-43bc-aab6-47f48bba4dcc", "content-length": "170", "date": "Wed, 30 Apr 2025 06:44:29 GMT"} })
169352025-04-30T06:44:30.434ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0/component/rot/caboose?firmware_slot=1
169362025-04-30T06:44:30.439ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "b1953ac2-d956-4da2-892f-c8050b7efcff", "content-length": "170", "date": "Wed, 30 Apr 2025 06:44:29 GMT"} })
169372025-04-30T06:44:30.439ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0/component/stage0/caboose?firmware_slot=0
169382025-04-30T06:44:30.445ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "a6254471-6534-4bd7-8215-566a655674e9", "content-length": "179", "date": "Wed, 30 Apr 2025 06:44:29 GMT"} })
169392025-04-30T06:44:30.445ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0/component/stage0/caboose?firmware_slot=1
169402025-04-30T06:44:30.457ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "eef7104f-0541-4eb9-98d9-b27abadb2c34", "content-length": "179", "date": "Wed, 30 Apr 2025 06:44:29 GMT"} })
169412025-04-30T06:44:30.457ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0/component/rot/cmpa
169422025-04-30T06:44:30.458ZDEBGtest_utilization: attempt to fetch latest inventory collection result = Ok("found")
169432025-04-30T06:44:30.460ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "2cce497b-6da6-4a7c-a098-a3a26022c224", "content-length": "702", "date": "Wed, 30 Apr 2025 06:44:29 GMT"} })
169442025-04-30T06:44:30.460ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0/component/rot/cfpa
169452025-04-30T06:44:30.462ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "5fda9bff-b52e-456b-bf31-0d309d22cfc5", "content-length": "727", "date": "Wed, 30 Apr 2025 06:44:29 GMT"} })
169462025-04-30T06:44:30.462ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0/component/rot/cfpa
169472025-04-30T06:44:30.465ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "0f90c052-ccfd-4b59-b0e4-9f921fc9d68c", "content-length": "729", "date": "Wed, 30 Apr 2025 06:44:29 GMT"} })
169482025-04-30T06:44:30.465ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/0/component/rot/cfpa
169492025-04-30T06:44:30.467ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "fc265223-652a-408c-bb57-0838dc3ec5f5", "content-length": "728", "date": "Wed, 30 Apr 2025 06:44:29 GMT"} })
169502025-04-30T06:44:30.468ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1
169512025-04-30T06:44:30.473ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1", status: 200, headers: {"content-type": "application/json", "x-request-id": "b751c093-d5ea-44c7-aefb-9b041f158c28", "content-length": "734", "date": "Wed, 30 Apr 2025 06:44:29 GMT"} })
169522025-04-30T06:44:30.473ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1/component/sp/caboose?firmware_slot=0
169532025-04-30T06:44:30.476ZWARNtest_utilization (results-sink-cluster): failed to insert some results into metric DB collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 error = Failed to check out connection to database: No backends found for this service file = oximeter/collector/src/results_sink.rs:92
169542025-04-30T06:44:30.477ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "dd35364f-ef10-482c-8571-34b0798a678a", "content-length": "111", "date": "Wed, 30 Apr 2025 06:44:29 GMT"} })
169552025-04-30T06:44:30.477ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1/component/sp/caboose?firmware_slot=1
169562025-04-30T06:44:30.484ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "51d2924d-8202-40ce-ae3e-5510d86cf4b2", "content-length": "111", "date": "Wed, 30 Apr 2025 06:44:29 GMT"} })
169572025-04-30T06:44:30.484ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1/component/rot/caboose?firmware_slot=0
169582025-04-30T06:44:30.486ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "c0c64f44-53fe-49c8-b6de-4cac05a7fa2d", "content-length": "170", "date": "Wed, 30 Apr 2025 06:44:29 GMT"} })
169592025-04-30T06:44:30.486ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1/component/rot/caboose?firmware_slot=1
169602025-04-30T06:44:30.495ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "eda8dfc6-3abb-43f1-a9e5-5ebcf45d98d5", "content-length": "170", "date": "Wed, 30 Apr 2025 06:44:29 GMT"} })
169612025-04-30T06:44:30.495ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1/component/stage0/caboose?firmware_slot=0
169622025-04-30T06:44:30.503ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "2343ae8a-2c49-4247-b5bd-06d1064f0a20", "content-length": "179", "date": "Wed, 30 Apr 2025 06:44:29 GMT"} })
169632025-04-30T06:44:30.503ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1/component/stage0/caboose?firmware_slot=1
169642025-04-30T06:44:30.507ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "4361fb1f-730b-473f-a7e2-79c9c99d9178", "content-length": "179", "date": "Wed, 30 Apr 2025 06:44:29 GMT"} })
169652025-04-30T06:44:30.507ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1/component/rot/cmpa
169662025-04-30T06:44:30.508ZTRACtest_utilization (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory
169672025-04-30T06:44:30.508ZTRACtest_utilization (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
169682025-04-30T06:44:30.508ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
169692025-04-30T06:44:30.508ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
169702025-04-30T06:44:30.509ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
169712025-04-30T06:44:30.509ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "edf68b4e-9483-4e5e-a490-80f03aca37af", "content-length": "702", "date": "Wed, 30 Apr 2025 06:44:29 GMT"} })
169722025-04-30T06:44:30.510ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1/component/rot/cfpa
169732025-04-30T06:44:30.512ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "33a14d0d-cd4a-4b4b-a545-4158bebccdf7", "content-length": "727", "date": "Wed, 30 Apr 2025 06:44:29 GMT"} })
169742025-04-30T06:44:30.512ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1/component/rot/cfpa
169752025-04-30T06:44:30.513ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
169762025-04-30T06:44:30.545ZDEBGtest_utilization (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory result = Ok(())
169772025-04-30T06:44:30.545ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
169782025-04-30T06:44:30.545ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
169792025-04-30T06:44:30.545ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
169802025-04-30T06:44:30.549ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "7cfc9b9a-74a6-450a-a461-6bdb4fc5a031", "content-length": "729", "date": "Wed, 30 Apr 2025 06:44:29 GMT"} })
169812025-04-30T06:44:30.550ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/switch/1/component/rot/cfpa
169822025-04-30T06:44:30.554ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
169832025-04-30T06:44:30.554ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
169842025-04-30T06:44:30.554ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
169852025-04-30T06:44:30.556ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "fb41b3a3-ec43-4625-ac40-8adf4d8286a4", "content-length": "728", "date": "Wed, 30 Apr 2025 06:44:29 GMT"} })
169862025-04-30T06:44:30.556ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0
169872025-04-30T06:44:30.560ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "ede09f23-044d-499d-8bc8-302fabce7d5f", "content-length": "723", "date": "Wed, 30 Apr 2025 06:44:29 GMT"} })
169882025-04-30T06:44:30.560ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0/component/sp/caboose?firmware_slot=0
169892025-04-30T06:44:30.565ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "483d2030-e632-4ce8-a3ba-7953ef5380e0", "content-length": "109", "date": "Wed, 30 Apr 2025 06:44:29 GMT"} })
169902025-04-30T06:44:30.565ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0/component/sp/caboose?firmware_slot=1
169912025-04-30T06:44:30.606ZDEBGtest_utilization: attempt to fetch latest inventory collection result = Ok("found")
169922025-04-30T06:44:30.612ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "6d6b7afb-a7ec-40b4-8551-96d362da21b7", "content-length": "109", "date": "Wed, 30 Apr 2025 06:44:29 GMT"} })
169932025-04-30T06:44:30.613ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0/component/rot/caboose?firmware_slot=0
169942025-04-30T06:44:30.628ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "9ca3af86-efad-4c55-93d3-768faa4aeb9a", "content-length": "169", "date": "Wed, 30 Apr 2025 06:44:29 GMT"} })
169952025-04-30T06:44:30.628ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0/component/rot/caboose?firmware_slot=1
169962025-04-30T06:44:30.632ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "72324b7d-2635-4352-8801-7a7de79a04b9", "content-length": "169", "date": "Wed, 30 Apr 2025 06:44:29 GMT"} })
169972025-04-30T06:44:30.632ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0/component/stage0/caboose?firmware_slot=0
169982025-04-30T06:44:30.634ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "f3e2664d-c954-4db4-99f5-f75a0acf3bd1", "content-length": "178", "date": "Wed, 30 Apr 2025 06:44:29 GMT"} })
169992025-04-30T06:44:30.635ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0/component/stage0/caboose?firmware_slot=1
170002025-04-30T06:44:30.637ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "45ffb613-abc6-4517-b33d-8bf22a18cd19", "content-length": "178", "date": "Wed, 30 Apr 2025 06:44:29 GMT"} })
170012025-04-30T06:44:30.637ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0/component/rot/cmpa
170022025-04-30T06:44:30.639ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "04c75a65-ac85-4b2f-996c-7f5fbc6d388d", "content-length": "702", "date": "Wed, 30 Apr 2025 06:44:29 GMT"} })
170032025-04-30T06:44:30.639ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0/component/rot/cfpa
170042025-04-30T06:44:30.641ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "ab78d026-4a7a-417e-a5b7-2ffb0c7159c1", "content-length": "727", "date": "Wed, 30 Apr 2025 06:44:29 GMT"} })
170052025-04-30T06:44:30.641ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0/component/rot/cfpa
170062025-04-30T06:44:30.644ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "ddae890f-f8b3-4b06-8505-509d218627f6", "content-length": "729", "date": "Wed, 30 Apr 2025 06:44:29 GMT"} })
170072025-04-30T06:44:30.644ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/0/component/rot/cfpa
170082025-04-30T06:44:30.647ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "b61fdc09-eafa-4d77-84b6-9e460dcc4509", "content-length": "728", "date": "Wed, 30 Apr 2025 06:44:29 GMT"} })
170092025-04-30T06:44:30.648ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1
170102025-04-30T06:44:30.650ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1", status: 200, headers: {"content-type": "application/json", "x-request-id": "98119a4d-e0af-4f04-947e-bda4776b41e4", "content-length": "723", "date": "Wed, 30 Apr 2025 06:44:29 GMT"} })
170112025-04-30T06:44:30.651ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1/component/sp/caboose?firmware_slot=0
170122025-04-30T06:44:30.654ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "0c3d0de5-b1a6-47ae-8044-85bcc4ff65c1", "content-length": "109", "date": "Wed, 30 Apr 2025 06:44:29 GMT"} })
170132025-04-30T06:44:30.655ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1/component/sp/caboose?firmware_slot=1
170142025-04-30T06:44:30.657ZTRACtest_utilization (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory
170152025-04-30T06:44:30.657ZTRACtest_utilization (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
170162025-04-30T06:44:30.657ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
170172025-04-30T06:44:30.658ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
170182025-04-30T06:44:30.661ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
170192025-04-30T06:44:30.666ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
170202025-04-30T06:44:30.680ZDEBGtest_utilization (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory result = Ok(())
170212025-04-30T06:44:30.680ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
170222025-04-30T06:44:30.681ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
170232025-04-30T06:44:30.685ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
170242025-04-30T06:44:30.688ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "821d2a74-9cb4-40da-b3b2-1694cfc02907", "content-length": "109", "date": "Wed, 30 Apr 2025 06:44:29 GMT"} })
170252025-04-30T06:44:30.688ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1/component/rot/caboose?firmware_slot=0
170262025-04-30T06:44:30.689ZTRACtest_utilization (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
170272025-04-30T06:44:30.689ZDEBGtest_utilization (InternalApi): roles roles = RoleSet { roles: {} }
170282025-04-30T06:44:30.690ZDEBGtest_utilization (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
170292025-04-30T06:44:30.693ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "cb1b56f6-ca67-477b-834b-b191dce42bb9", "content-length": "169", "date": "Wed, 30 Apr 2025 06:44:29 GMT"} })
170302025-04-30T06:44:30.693ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1/component/rot/caboose?firmware_slot=1
170312025-04-30T06:44:30.698ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "2d6b08b6-747b-4493-9e24-e741ab1fbbd0", "content-length": "169", "date": "Wed, 30 Apr 2025 06:44:29 GMT"} })
170322025-04-30T06:44:30.698ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1/component/stage0/caboose?firmware_slot=0
170332025-04-30T06:44:30.704ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "fe912339-bd49-4efb-a0bb-e9af28f7ca08", "content-length": "178", "date": "Wed, 30 Apr 2025 06:44:29 GMT"} })
170342025-04-30T06:44:30.704ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1/component/stage0/caboose?firmware_slot=1
170352025-04-30T06:44:30.709ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "926ab1dc-9d40-4ddd-bd06-4721a44510a5", "content-length": "178", "date": "Wed, 30 Apr 2025 06:44:29 GMT"} })
170362025-04-30T06:44:30.710ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1/component/rot/cmpa
170372025-04-30T06:44:30.711ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "db3f039b-e336-4b3f-ad61-6f098374cf71", "content-length": "702", "date": "Wed, 30 Apr 2025 06:44:29 GMT"} })
170382025-04-30T06:44:30.711ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1/component/rot/cfpa
170392025-04-30T06:44:30.714ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "8ffe56c3-a2a7-4eac-bc82-17e1baef0e42", "content-length": "727", "date": "Wed, 30 Apr 2025 06:44:29 GMT"} })
170402025-04-30T06:44:30.714ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1/component/rot/cfpa
170412025-04-30T06:44:30.717ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "2c25375a-784b-4c51-badd-9140965b3b56", "content-length": "729", "date": "Wed, 30 Apr 2025 06:44:29 GMT"} })
170422025-04-30T06:44:30.718ZDEBGtest_utilization (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:48759 method = GET uri = http://[::1]:48759/sp/sled/1/component/rot/cfpa
170432025-04-30T06:44:30.719ZDEBGtest_utilization (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:48759 result = Ok(Response { url: "http://[::1]:48759/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "f3a66f65-f7a6-4a50-a80b-6fd919184f5d", "content-length": "728", "date": "Wed, 30 Apr 2025 06:44:29 GMT"} })
170442025-04-30T06:44:30.719ZTRACtest_utilization (BackgroundTasks): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet
170452025-04-30T06:44:30.719ZTRACtest_utilization (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
170462025-04-30T06:44:30.719ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
170472025-04-30T06:44:30.719ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
170482025-04-30T06:44:30.720ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
170492025-04-30T06:44:30.732ZDEBGtest_utilization: attempt to fetch latest inventory collection result = Ok("found")
170502025-04-30T06:44:30.738ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
170512025-04-30T06:44:30.738ZDEBGtest_utilization (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet result = Ok(())
170522025-04-30T06:44:30.738ZTRACtest_utilization (BackgroundTasks): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet
170532025-04-30T06:44:30.738ZTRACtest_utilization (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
170542025-04-30T06:44:30.738ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
170552025-04-30T06:44:30.738ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
170562025-04-30T06:44:30.739ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
170572025-04-30T06:44:30.743ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
170582025-04-30T06:44:30.754ZDEBGtest_utilization (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet result = Ok(())
170592025-04-30T06:44:30.755ZTRACtest_utilization (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
170602025-04-30T06:44:30.755ZDEBGtest_utilization (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
170612025-04-30T06:44:30.756ZDEBGtest_utilization (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
170622025-04-30T06:44:30.840ZDEBGtest_utilization (BackgroundTasks): begin collection from Sled Agent background_task = inventory_collection sled_agent_url = http://[::1]:39365
170632025-04-30T06:44:30.840ZDEBGtest_utilization (BackgroundTasks): client request SledAgent = http://[::1]:39365 background_task = inventory_collection body = None method = GET uri = http://[::1]:39365/inventory